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1.  Introduction 


This  document  describes  research  conducted  by  SRI  International  on  distributed 
artificial  intelligence  (DAI)  under  ONR  Contract  N00014-80-C-0296  during  the  period  February 
1,  1980  to  June  30,  1982.  In  addition  to  an  overview  describing  the  general  goals  of  the  DAI 
project  and  the  major  subareas  covered,  this  document  includes  three  papers  describing  several 
facets  of  our  research  in  greater  detail. 

''  The  ultimate  goal  of  the  DAI  project  is  to  discover  the  essential  principles  upon  which 
construction  of  a  network  of  cooperative  intelligent  computer  agents  could  be  based.  The 
motivations  for  distributed  AI  arise  from  at  least  three  sources.  First,  many  military  problems 
have  an  inherently  distributed  character  by  virtue  of  the  physical  distribution  not  only  of 
military  units  themselves  but  also  of  the  various  sensors  that  provide  information  affecting 
those  units.  Second,  the  changing  economies  of  computer  hardware  and  communications 
technology  are  making  distributed  systems  increasingly  cost-effective;  since  distributed  systems 
with  computationally  powerful  processors  will  soon  be  more  commonplace,  it  is  worthwhile 
considering  how  AI  could  be  used  to  exploit  such  systems  more  fully.  Third,  there  is  a  strong 
motivation  from  within  AI.  Artificial-intelligence  systems  that  act  in  an  environment  must 
reason  about  that  environment.  Thus,  when  the  environment  contains  other  agents,  as  most 
realistic  environments  do,  the  AI  system  must  reason  about  the  state  of  those  agents  and  how 
to  affect  it.  This  leads  naturally  to  distributed  AI.  ^ — 

The  first  years  of  the  DAI  project  at  SRI  have  been  devoted  to  the  basic  research  issues 
connected  with  knowledge  representation  and  planning  in  domains  containing  multiple  agents. 
Our  framework  assumes  a  system  containing  physical  objects  and  a  number  of  “agents.”  Each 
agent  is  endowed  with  sets  of  beliefs,  goals,  and  intentions  (plans)  operationalized  as  expressions 
in  suitable  language(s).  These  beliefs,  goals,  and  intentions  constitute  the  “cognitive  state”  of 
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the  agent,  and  the  overall  state  of  the  system  consists  of  the  physical  state  of  the  objects 
combined  with  the  cognitive  states  of  the  agents.  The  agents  are  capable  of  performing  actions 
(or  “operations” ),  and  the  next  state  of  the  system  depends  on  the  actions  performed  by  the 
several  agents. 

As  with  traditional  robot  problem-solving  systems,  the  choice  of  operators  determines 
what  state  transitions  can  occur.  In  classical  single-agent  problem  solving,  the  operators 
typically  affect  only  the  physical  state  of  the  system.  In  contrast  thereto,  we  are  primarily 
interested  in  operators  whose  effect  is  to  change  the  cognitive  state  of  other  agents,  both  as 
a  goal  in  itself  and  as  a  means  toward  achieving  some  “physical”  goal  by  enlisting  the  aid  of 
another  agent. 

From  each  agent’s  point  of  view,  the  planning  problem  consists  of  forming  an  intention 
(plan)  over  the  operators  to  achieve  its  goals,  given  what  it  believes.  The  difficulty  posed  by  this 
problem  in  the  distributed  case  stems  directly  from  the  complexity  of  cognitive-state-affecting 
operators.  We  have  been  looking  at  various  formalisms  for  describing  cognitive  state  that  would 
be  suitable  for  practical  systems  of  the  type  described.  In  addition,  we  have  been  investigating 
useful  sets  of  operators  for  achieving  interagent  cooperation. 

1.1.  Planning  in  a  World  with  Other  Agents 

Most  AI  planning  research  to  date  has  been  based  on  a  simple  state-transition  model 
of  action  in  which  there  is  only  one  agent  and  whose  actions  are  always  determinate.  To  handle 
complex  domains  realistically,  we  need  to  extend  the  modet  to  allow  actions  with  indeterminate 
outcomes  (especially  when  outcomes  differ  in  likelihood),  as  well  as  actions  by  more  than  one 
agent. 

There  are  several  ways  of  adding  indeterminacy  to  the  underlying  framework.  The 
situation  calculus  formulation  of  planning  is  able  to  express  indeterminacy  up  to  logical  dis¬ 
junction  by  simply  having  the  axioms  that  express  the  effects  of  an  action  contain  disjunctive 
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postconditions.  Unfortunately,  the  STRIPS  formulation,  which  suppresses  state  variables  and 
expresses  the  effects  of  actions  as  state-description  transformations,  is  incapable  of  expressing 
this  indeterminacy.  Our  work  on  dynamic-logic-based  planning  addresses  this  problem  by 
combining  some  of  the  best  features  of  STRIPS  (e.g.  the  suppression  of  state  variables  and  the 
use  of  structured  search  through  a  space  of  state  descriptions)  with  the  best  features  of  the 
situation  calculus  (e.g.,  the  possibility  of  disjunctive  postconditions).  Some  results  of  this  work 
are  presented  in  Chapter  2. 

The  inclusion  of  concurrency  in  the  formal  model  can  be  handled  with  parameteriza¬ 
tion  of  the  state  transformations  by  the  actions  of  several  agents  instead  of  only  one.  If  we 
are  willing  to  postulate  a  global  state  of  the  system  (even  though  in  practice  we  may  have  only 
incomplete  knowledge  of  this  state),  we  can  conceive  of  the  parallel  execution  of  a  primitive 
operation  as  being  a  single  complex  operation  on  the  global  state.  Reasoning  about  sequences 
of  actions  by  the  various  agents  then  involves  reasoning  about  interleavings  of  primitive  events. 
This  would  be  difficult  if  the  only  way  to  perform  this  reasoning  were  to  enumerate  the  com- 
binatorially  large  number  of  execution  sequences  and  examine  each  in  turn.  Fortunately,  in 
typical  domains  the  effects  of  an  action  by  one  agent  are  ordinarily  invariant  under  most  ac¬ 
tions  by  other  agents.  This  fact  can  be  used  to  facilitate  the  reasoning.  We  have  investigated 
extensions  of  the  planning  framework  that  include  this  form  of  concurrency. 

1.2.  Reasoning  about  the  Knowledge  of  Other  Agents 

Part  of  the  problem  of  reasoning  about  the  world  involves  reasoning  about  other 
agents  in  the  world.  Part  of  what  we  know  about  other  agents  has  to  do  with  how  they  act, 
especially  how  they  act  in  light  of  their  knowledge  and  desires.  In  brief,  it  is  often  useful  to 
simplify  our  description  of  the  behavior  of  agents  by  attributing  to  them  certain  beliefs  and 
goals  which,  with  the  presumption  of  rationality,  predispose  them  toward  certain  actions  that 
would  tend  to  further  their  goals  if  the  world  were  in  accord  with  their  beliefs.  Thus,  an  AI 
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system  that  would  use  the  principle  of  rationality  to  predict  the  actions  of  other  agents  would 
need  to  be  able  to  model  those  agents’  beliefs. 


One  method  for  modeling  the  beliefs  of  other  agents  involves  using  modal  logic — i.e., 
a  logic  with  special  operators  such  as  BELIEVB(p),  where  p  is  a  sentence,  possibly  involving  the 
BELIEVE  operator  as  well.  The  semantics  of  this  operator  is  often  given  in  terms  of  possible 
worlds.  That  is,  what  we  mean  when  we  say  that  “x  believes  p”  is  that  “p  is  true  in  all  worlds 
compatible  with  what  x  believes.”  We  note  that  the  object  of  belief  is  no  longer  an  entity  in 
the  logic.  For  example,  we  cannot  quantify  over  beliefs.  Another  possible  limitation  of  the 
modal  approach,  at  least  in  its  usual  form,  is  that  it  commits  the  agent  to  believing  all  the 
consequences  of  his  beliefs.  This  assumption  is  often  a  useful  idealisation,  but  is  not  always 
appropriate  for  modeling  resource- bounded  processes  like  A1  problem-solving  systems. 

One  alternative  to  the  modal  approach  is  often  called  the  “syntactic”  approach.  In 
this  method  the  object  of  belief  is  taken  to  be  roughly  a  sentence  or  a  formula  of  some  logic. 
Many  objections  have  been  raised  to  this  approach,  chiefly  centering  around  the  treatment 
of  quantification.  However,  a  careful  formulation  of  the  logic  not  only  avoids  most  of  these 
problems,  but  seems  to  have  the  advantage  of  being  convenient  to  implement.  Some  DAI-related 
results  by  Kurt  Konolige  in  this  area  are  reported  in  Chapter  3. 


1.3.  Planning  Communication  Acts 

One  view  of  communication  is  that  it  involves  sending  messages  that  are  “dumps”  of 
part  of  the  sender’s  mental  state.  In  other  words,  the  message  “encodes”  some  of  the  beliefs  of 
the  speaker,  and  all  the  hearer  has  to  do  is  “decode”  the  message,  adding  the  beliefs  to  his  own 
belief  set.  A  more  sophisticated  view  holds  that  messages  are  carefully  built  with  the  intent  to 
affect  the  state  of  the  hearer  in  ways  far  more  subtle  than  simply  adding  a  belief  to  the  hearer’s 
set  of  beliefs.  For  instance,  in  an  indirect  communication  act,  the  utterance  “it  is  cold  in  this 
room"  could  be  recognized  as  a  request  to  turn  up  the  heat. 


To  use  messages  of  this  complexity,  it  is  necessary  for  an  AI  system  to  contain  both  a 
model  of  the  effects  of  various  communication  acts  and  a  procedure  for  synthesizing  utterances 
that  achieve  desired  effects.  During  the  past  two  years  Doug  Appelt  has  investigated  this 
question.  The  result  described  in  his  recent  Ph.D.  thesis  is  a  system  for  planning  utterances 
[App82].  A  condensed  version  of  these  results  are  presented  as  Chapter  4. 

1.4.  Future  Plans 

During  the  next  year  we  hope  to  extend  our  theoretical  understanding  of  multiagent 
planning  and  develop  algorithms  for  consensus  formation  and  plan  sharing.  We  also  intend 
to  implement  a  planning  system  that  is  capable  of  utilizing  other  similar  agents  to  achieve 
shared  goafs.  Although  we  expect  the  principles  underlying  this  planner  to  be  largely  domain- 
independent  we  shall  select  a  particular  domain  for  experimentation.  We  will  continue  our 
attempts  to  characterize  a  variety  of  communication-act  types  using  cognitive-state  logics,  and, 
to  the  extent  possible,  we  shall  integrate  the  results  into  the  evolving  implementation. 


2.  Plan  Synthesis:  A  Logical  Perspective 


This  section  was  written  by  Stanley  Rosenschein. 

Abstract 

This  paper  explores  some  theoretical  issues  of  robot  planning  from  the  perspective 
of  propositional  dynamic  logic.  A  generalized  notion  of  “progression”  and  “regression”  of 
conditions  through  actions  is  developed.  This  leads  to  a  bidirectional  single-level  planning 
algorithm  which  is  easily  extended  to  hierarchical  planning.  Multiple  pre/post-condition  pairs, 
complex  (conjunctive,  disjunctive)  goals,  goals  of  maintenance  and  prevention,  and  plans  with 
tests  are  all  handled  in  a  natural  way.  The  logical  framework  is  used  to  clarify  gaps  in  existing 
“no-,  -iear”  and  “hierachical”  planning  strategies. 

2.1.  Introduction 

Although  the  connection  between  the  A. I.  planning  problem  and  automated  program 
synthesis  is  widely  acknowledged,  relatively  little  planning  research  has  made  explicit  use  of 
concepts  from  the  logic  of  programs.  Such  logics,  however,  offer  theoretical  insight  into  various 
issues  in  A.I.  planning,  including  compound  goals  and  levels  of  abstraction  [Sac75,  Sac77, 
Tat77,Hay79].  Many  of  these  issues  arise  in  their  purest  form  in  domains  describable  in  the 
propositional  calculus  (e.g.  simple  blocks  worlds)  as  evidenced  by  the  literature  on  the  subject 
[Sus75,  Sac75,  Wal75j.  Thus  for  clarity  and  continuity,  we  choose  the  propositional  setting  to 
develop  a  unified,  abstract  treatment  of  these  issues  using  propositional  dynamic  logic  (PDL) 
as  our  primary  logical  tool  (Pra76,  Lit77,  Pra78a,  Pra78b,  Har79j. 


PDL  is  a  decidable  modal  propositional  logic  for  reasoning  about  binary  state-relations 
induced  by  programs.  In  theory,  the  existence  of  such  a  logic  provides  an  immediate  “solution” 
to  the  propositional  planning  problem:  One  could  systematically  substitute  all  possible  plans 
into  a  schema  (the  specification)  which  asserts  the  desired  property  of  the  plan.  The  resulting 
expressions  could  be  tested  for  validity  to  filter  out  non-solutions.  Unfortunately,  this  fact  is 
of  little  practical  consequence,  as  such  a  procedure  is  certain  to  be  grossly  inefficient.  The 
approach  developed  here  imposes  additional  structure  by  (1)  considering  a  class  of  problems 
that  require,  in  effect,  only  non-modal  reasoning  and  (2)  using  suitable  “progression”  and 
“regression”  operators  to  structure  the  search  for  a  solution  and  allow  early  pruning  of  hopeless 
paths. 


Surprisingly,  even  our  restricted  formulation  covers  a  more  general  class  of  problems 
than  are  handled  by  most  comparable  A.I.  planning  methods.  For  instance,  we  allow  goals 
to  be  arbitrary  wffs,  so  disjunctive  goals  (cited  as  an  unsolved  problem  by  Sacerdoti  [Sac77]) 
require  no  special  treatment  at  all.  The  approach  provides  a  theoretical  basis  for  hierarchical 
plan  generation  that  ties  in  directly  with  current  ideas  on  hierarchical  program  development 
(see  section  1.3.3).  In  addition,  the  use  of  program  logic  provides  a  formal  basis  for  specifying 
and  verifying  the  plan-generating  system  itself. 


Although  our  work  can  be  generalized  along  several  dimensions  (propositional  axiom 
schemata,  plans  with  loops,  quantified  pre/ post-conditions,  etc.),  these  are  beyond  the  scope  of 
the  current  paper,  which  focuses  instead  on  the  essential  structure  of  the  approach.  At  the  same 
time,  it  should  be  noted  that  the  use  of  axiom  schemata  seems  to  be  a  minimal  requirement 
for  a  practical  application.  This  paper  should  be  regarded  as  a  foundational  study  aimed 
at  deepening  our  understanding  of  planning;  a  separate  paper  will  describe  implementation 
considerations  [Ros81j. 
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2.2.  Preliminaries 


This  section  briefly  presents  the  basic  concepts  of  a  loop-free  fragment  of  propositional 
dynamic  logic  (PDL).  The  interested  reader  is  referred  to  [Pra76,  Lit77,  Har79]  for  a  more 
thorough  treatment  of  dynamic  logic  in  its  entirety.  Though  dynamic  logic  is  ordinarily  used 
to  reason  about  program s,  it  is  equally  appropriate  for  reasoning  about  plant  (in  the  A. I.  sense); 
thus  in  this  paper  the  terms  program  and  plan  are  used  interchangeably. 


2.2.1  Syntax 

Let  P  and  A  denote  two  symbol  sets:  atomic  propositions  and  atomic  actions,  respec¬ 
tively.  Define  wffs  Pp,a  and  programs  Apt&  simultaneously  (deleting  subscripts  for  con¬ 
venience): 

(1)  PCP 

(2)  A  C  A 

(3)  If  p,q  €  P,  then -p,p  V  q€  P 

(4)  If  p  €  P  and  a  6  A,  then  <a>p  €  P 

(5)  A  €  A 

(6)  If  p  6  P  and  p  is  nonmodal  (see  below),  then  pi  E  A 

(7)  If  a,  /?  6  A,  then  a;fl  €  A  and  or  U  0  €  A 

A  formula  is  nonmodal  if  it  contains  no  subformula  of  the  form  <a>p.  We  abbreviate 
->(->p  V  ->g)  as  p  A  q,  “>p  V  q  as  p  D  q,  (p  D  q)  A  {q  D  p)  as  p  ss  q,  p  V  -'p  as  true,  and 
p  A  ~>p  as  false.  Parentheses  are  used  when  necessary  according  to  the  usual  conventions. 


2.2.2  Semantics 

A  structure  S  is  a  triple  {W,  n,  m)  where  W  is  a  non-empty  set  of  “worlds,”  n  :  P  — 
2W ,  and  m  :  A  — *  2Wx  w .  That  is,  it  assigns  to  each  atomic  proposition  p  the  subset  of  W  where 
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p  holds,  and  m  assigns  to  each  atomic  action  a  the  binary  relation  over  W  representing  the 
next-state  relation  for  a.  Given  a  structure  S,  meanings  can  be  assigned  to  arbitrary  programs 
and  formulae  by  extending  m  and  it: 

Meanings  of  Programs 

(1)  m(A)  =  {(«,  s)|s  6  IF)  (identity  relation  over  W) 

(2)  m(p?)  =  {(«,  «)|«  6  Jr(p)}  (identity  relation  restricted  to  worlds  where  p  holds) 

(3)  m(o;  0)  =  m( a)  °  m(0)  (composition  of  relations) 

(4)  m(o  U  0)  —  m(or)  U  m(0)  (anion  of  relations  considered  as  sets) 

Meanings  of  Formulae 

(1)  it[^p)^w-Ap) 

(2)  ir{p  V  q)  —  ir[p)  U  jr(g) 

(3)  jr(<a>p)  =  {«  6  VF|  3t  6  VF.(«,t)  6  m(a)  A  t  6  >Kp)} 

The  last  equation  asserts  that  <a>p  is  true  in  those  worlds  «  from  which  another 
world  t  is  reachable  via  a’s  next-state  relation  such  that  p  holds  in  t.  In  general  our  formulas 
will  involve  the  dual  of  <a>,  namely  [a].  [a]p  can  be  read  “after  a,  p.”  The  intent  is  that  p 
holds  in  all  worlds  accessible  via  a. 

A  formula  p  is  valid  in  a  structure  S  =“  (W,  it,  m)  (written  S  p)  iff  it{p)  *■»  W;  p  is 
valid  (written  f—  p)  iff  it  is  valid  in  every  structure. 

2.2.3  Axiomatics 

The  following  system  captures  the  semantics  given  in  the  previous  section: 

Axioms 

(1)  Axioms  of  the  propositional  calculus 

(2)  (o)(p  Dq)D  ([a]p  D  (0)9) 

(3)  (Ajp  =  p 
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(4)  [p?]g=pDg 

(5)  [a;p]p  =  (a][£jp 

(6)  [a  U  P)p  m  [a]p  A  [/ 3]p 

Rules  of  Inference 

(1)  From  p,  p  D  q  derive  q  (Modus  Ponens) 

(2)  From  p  derive  [a]p  (Necessitation) 

If  a  formula  p  follows  from  these  axioms  under  the  stated  rules  of  inference,  we  say  it 
is  provable  and  write  | —  p;  if  p  can  be  proved  from  a  set  of  assumptions,  Q,  we  write  Q\—p. 


2.2.4  A  Restricted  Class  of  Programs 

PDL  breaks  the  ordinary  conditional  statement  into  more  primitive  notions  of  “test” 
(?)  and  “non-deterministic  choice”  (U).  Though  we  allow  the  primitive  actions  to  be  non- 
deterministic,  we  shall  only  be  interested  in  deterministic  combining  forms.  Thus  we  limit 
the  use  of  ?  and  U  to  contexts  of  the  form  (p?;a)  U  (->p?;/3)  and  require  (for  convenience 
only)  that  p  be  atomic.  This  corresponds  to  the  ordinary  conditional,  so  we  abbreviate  this 
program  form  to  p  -*  a,  ft  and  call  the  class  of  programs  obeying  these  syntactic  restrictions 
C-programt  (symbolically  A?, a).  The  requirement  that  p  be  atomic  is  not  restrictive,  since 
arbitrary  boolean  combinations  of  tests  can  be  expressed  by  appropriate  use  of  (possibly  nested) 
conditionals.  For  example,  ->p  — >  a,  ft  is  equivalent  to  p  — ►  ft,  a.  (p  A  q)  —  a,  ft  is  equivalent 
to  (p  — *  (q  — *  a,  ft),  ft),  and  so  forth. 

An  important  property  of  our  combining  forms  is  that  they  preserve  termination;  if 
the  primitives  always  terminate,  every  C-program  over  those  primitives  will  always  terminate. 
(Loops  are  conspicuously  absent.)  In  PDL,  the  fact  that  a  program  a  always  terminates  is 
expressed  <a>true. 
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2.3.  A  Planning  Method 


Having  described  a  suitable  language  and  logic,  we  are  now  in  a  position  to  discuss 
planning  methods.  Section  1.3.1  contains  the  formal  definition  of  a  (single-level)  “planning 
problem”  and  the  corresponding  notion  of  a  “solution.”  This  leads  directly  (section  1. 3.2.2) 
to  a  bidirectional  (single-level)  planning  algorithm  based  on  “progressing”  and  “regressing” 
conditions  through  actions.  Section  1.3.3  describes  how  the  hierarchical  planning  problem  can 
be  regarded  as  a  succession  of  single-level  problems  in  a  way  that  makes  the  connection  between 
the  levels  logically  precise. 


2.3.1  Definitions 


A  planning  problem  is  a  triple  (V,Q,R(u))  where 


V  =  (P,A) 


Q 


R(u) 


is  the  vocabulary  of  the  problem,  consisting  of  the  atomic  propositions  and  the 
atomic  actions.1 

is  a  finite  set  of  axioms,  which  we  will  refer  to  as  domain  constraints.  Q  is 
partitioned  into  two  subsets:  static  constraints,  which  are  nonmodal  formulae, 
and  dynamic  constraints,  which  are  always  of  the  form  p  D  [a]?,  p  and  q 
being  arbitrary  nonmodal  wfis  and  a  an  atomic  action.  We  implicitly  assume 
an  axiom  of  the  form  <a>true  for  every  atomic  action  a;  this  expresses  the 
fact  that  the  action  a  always  terminates,  though  Q  may  only  partially  specify 
in  what  state  a  terminates.  We  also  assume  that  Q  is  consistent. 

is  a  finite  set  of  formulae  called  the  plan  constraints.  Like  the  dynamic  domain 
constraints,  each  of  these  is  of  the  form  p  D  [ujg  for  nonmodal  p  and  q.  The 
symbol  u  is  a  distinguished  atomic  action  not  contained  in  A. 


A  solution  to  a  planning  problem  ( V,Q,R(u ))  is  an  expression  a  in  the  programming 
language  Av  such  that  for  every  r(a)  (obtained  by  substituting  a  for  ti  in  R),  Q  )—  r(a). 


I.e.,  it  is  provable  from  the  domain  constraints  Q  that  a  satisfies  all  the  plan  constraints.2 


^or  some  applications  it  is  desirable  to  constrain  the  programming  language  to  use  only  a  designated  subset 
of  the  propositions  as  tests  in  conditionals.  This  requires  a  straightforward  modification  of  our  definition  and 
will  not  be  pursued  here. 

2 Equivalently  in  semantic  terms:  Structures  that  satisfy  the  domain  constraints  also  satisfy  the  a- instantiated 
plan  constraints. 
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(Because  of  the  termination  constraints  on  the  atomic  actions,  a  is  guaranteed  to  terminate. 
So  in  the  language  of  program  logic,  we  are  talking  about  “total  correctness.” ) 


2.3.2  Finding  Solutions 

Having  defined  “solutions,”  we  turn  our  attention  to  methods  for  discovering  them. 
A  natural  way  of  organizing  the  search  for  solutions  is  to  follow  the  syntactic  structure  of  the 
programming  language. 

Recall  that  a  program  is  either  A,  an  atomic  action  a,  or  a  composite  of  the  form  a;  $ 
or  t  —*  a,  fJ  where  a  and  $  are  programs  and  t  is  an  atomic  proposition.  It  will  simplify  the 
algorithm  to  consider  only  programs  in  a  normal  form,  which  we  now  define. 


2.3.2.1  Normal  Form  for  Conditional  Plans 

A  program  is  in  normal  form  if  it  consists  of  a  sequence3  of  0  or  more  atomic  actions 
followed  optionally  by  a  conditional  program  both  branches  of  which  are  in  normal  form, 
followed  in  turn  by  0  or  more  further  atomic  actions.  More  formally,  a  program  is  in  normal 
form  if  it  can  be  written  as  At; . .  .;An,  n  >  0,  with  at  most  one  A%  not  atomic,  in  which  case 
Ai  is  of  the  form  t  — *  Bi ,  Bz  where  both  Bj  are  themselves  in  normal  form.4  The  null  sequence 
is  identified  with  A,  and  we  take  A;  a  =>  a;  A  **  a. 

We  have  not  lost  any  essential  solutions  by  insisting  on  this  form  since  every  C- 
program  can  be  put  into  normal  form  by  transforming  the  longest  (length  >  1)  sequence  of 

3Since  •;*  is  associative,  we  write  sequences  a;  4; . . . ;  c  without  indicating  order  of  association. 

^Warren’s  method  [War76]  for  introducing  conditionals  produces  plans  of  an  even  more  restricted  form:  the 
conditional  must  be  the  l—t  action  in  the  sequence.  I.e.,  a  plan,  once  split,  may  never  rejoin.  This  is  not  an 
essential  limitation,  but  it  introduces  a  somewhat  greater  degree  of  redundancy  than  our  form.  We  note  in 
passing  that  Warren's  view  of  the  conditional  test  as  an  action,  has  much  in  common  with  PDL’s  p?  action. 
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steps  whose  first  and  last  steps  are  conditionals  into  a  single  conditional  as  follows: 

(s  -  A,  B); . . . ;  ( t  -  C,  D)  =*  (s  -  (A; . . . ;  (t  -  C,  D)),  -*■  C,  £>))) 

and  applying  this  transformation  recursively  to  A,  B,  and  the  residual  . . .  ;(t  — ►  C,D). 

2.3.2.2  An  Algorithm 

Suppose  we  are  looking  for  a  normal-form  program  a  that  satisfies  one  of  the  dynamic 
constraints  p  D  [0)9  in  R.  Consider  the  following  cases  corresponding  to  the  possible  forms  of 
a: 

(1)  a  =  A.  This  is  a  solution  if  Q  |—  p  D  q. 

(2)  a  =  a;  ft  or  a  =  ft;  a  for  some  atomic  action  a.  In  the  former  case,  a  is  a  solution  if 
Q  \—  p/ a  D  \ft\q,  where  p/a  represents  the  strongest  provable  post-condition  of  p  and 
a.  Analogously,  in  the  second  case,  a  is  a  solution  if  Q  |—  p  D  [/7]a\g  where  a\q  is  the 
weakest  provable  precondition  of  a  and  q.  We  call  the  former  case  “progression”  and  the 
latter  “regression.” 

(3)  a  =  t  — *  Pi,  fa-  In  this  case,  a  is  a  solution  if  Q  H  P  A  t  D  [Pi]q  and  Q  \—  p  A  -»f  D 

[fa\ <?• 

We  see  that  (1)  defines  success,  (2)  suggests  forward  and  backward  strategies  for 
sequential  steps,  and  (3)  suggests  a  forward  strategy  for  conditionals. 

In  addition,  we  see  that  there  are  several  obvious  ways  to  limit  the  search.  First,  if 
p  D  p/a,  the  forward  search  need  not  consider  action  a.  (A  special  case  of  this  arises  when 
p/a  =  true.)  Dually,  if  a\q  D  q,  the  backward  search  need  not  consider  a.  (Here  we  have  a 
special  case  when  a\q  —  false.)  These  checks  eliminate  self-loops.  We  can  eliminate  cycling 
in  the  search  space  altogether  if  we  are  willing  to  pay  the  price  of  checking  whether  p,-  D  p/a 
for  any  p,-  in  the  leading  chain  of  preconditions.  Likewise  we  can  check  whether  a\q  D  qj  for 
any  qj  in  the  trailing  chain  of  postconditions.  Note  also  that  if  p  D  t  or  p  D  ->t,  the  forward 
conditional  search  involving  t  need  not  be  pursued,  p/a  can  never  be  false  since  this  would 


imply  failure  of  a  to  terminate,  contradicting  our  assumptions  about  the  domain  constraints 

Q- 


These  observations  lead  immediately  to  the  following  non-deterministic  algorithm  for 
computing  solutions  for  the  single  constraint  p  D  [ajq:  (Multiple  constraints  will  be  discussed 
later.) 


Bigression  Algorithm 

“Bigression”  stands  for  “bidirectional  progression  and  regression.”  Assume  p,  q  are 
not  false. 

Solve(p,q)  =  Bigress(p,g,  A,  A). 

Bigresa (pre,post,  leader,  trailer): 

JF  Q  \—  pre  D  post  THEN  RETURN  (leader;  trailer). 

CHOOSE: 

CHOOSE  <a,pre/a>  from  LiveForward(pre): 

RETURN  (Bigress(pre/a,po*t,  leader;  a,  trailer )) 

CHOOSE  <a,a\post>  from  LiveBackward(post): 

RETURN  (Bigress(pre,  a\post,  leader,  a;  trailer )) 

CHOOSE  t  from  NonTriv(pr«): 

RETURN  (leader;  C;  trailer) 

where  C  (t  — *  Bigress(pre  A  t,  post,  A,  A),  Bigress(pre  A 

-»  t,  post,  A,  A)) 

LiveForward(p): 

IF  S  ^  <i> 

where  S  =  (<a,p/a>|a  €  A,Q  \/  p  D  p/a} 

THEN  RETURN  ( S ) 

ELSE  FAILO 

LiveBackward(q): 

IF  S  /  <t> 

where  S  =*  {<a,a\q>|a  €  A,Q  f/  a\q  D  q} 

THEN  RETURN  (S) 

ELSE  FAIL(). 

NonTriv(p): 

IF  5  4> 

where  5  -  {t|<  €  P,Q  \/  P  D  t,Q  \/  p  D  ->  t} 

THEN  RETURN  (5) 
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ELSE  FA1L0. 


The  algorithm  as  presented  finds  solutions  for  a  single  plan  constraint.  However  the 
extension  to  the  general  case  is  straightforward:  To  insure  that  all  the  plan  constraints  are  met, 
a  “Cartesian  product”  version  of  this  algorithm  must  be  run.  A  failure  in  any  of  the  constraint 
components  counts  as  failure  and  serves  to  prune  that  branch. 

The  bigression  algorithm  makes  use  of  three  additional  auxiliary  functions:  aQ 
“/”,  and  “\".  UQ  (— ' ”  is  a  procedure  which  takes  as  input  a  nonmodal  formulae  p  and  decides 
whether  p  is  provable  from  Q.  If  the  static  axioms,  Q„  are  rich  enough,6  this  check  can  be  done 
using  only  nonmodal  reasoning,  i.e.,  ordinary  propositional  decision  methods.  The  functions 
“/”  (progression)  and  “\”  (regression)  are  the  subject  of  the  next  section. 


2.3.2.3  Progression  and  Regression 


Ideally,  we  would  like  p/a  to  compute  the  strongest  postcondition  of  condition  p  and 
action  a.  Similarly,  we  would  like  a\q  to  compute  the  weakest  precondition  [Dij75,Wal75].  In 
PDL,  the  weakest  precondition  of  p  and  a  can  be  expressed  simply  as  [a]p,  which  is  obviously 
the  weakest  formula  implying  “after  a,  p”.  The  strongest  postcondition  can  be  expressed  using 
a  “converse”  operator  which  we  have  not  described.  (See  [Lit77].) 

However,  given  the  restricted  form  of  our  dynamic  axioms,  there  will  be  no  proposi¬ 
tional  formula  provably  equivalent  to  either  of  these  modal  formulae.  On  the  other  hand, 
we  can  effectively  compute  the  weakest  precondition  prc  and  strongest  postcondition  poet  for 
which  it  is  provable  from  Q  that  p re  implies  “after  a,  q”  and  p  implies  “after  a,  poet.”  It  is 
these  propositional  formulae  that  we  label  p/a  and  a\q. 


Specifically,  Q,  must  generate  all  the  nonmodal  formulae  generated  by  all  of  Q.  In  certain  pathological  cases, 
such  as  when  Q  contains  a  dynamic  axiom  of  the  form  p  3  [a]/al$e,  Q  would  hare  to  be  extended  to  include 
extra  static  axioms,  since  p  3  \a\fahe  and  <a>true  together  imply  ->p  —  a  nonmodal  formula  derirable 
only  through  modal  reasoning. 


The  formula  p/a  is  found  by  taking  the  conjunction  of  the  set  of  formulae  each  of 
which  is  a  disjunction  of  a  set  of  <7,-  drawn  from  the  “right  hand  side”  of  the  dynamic  axioms  of 
Q  ( pi  D  such  that  the  disjunction  of  the  corresponding  p,’s  is  implied  by  p.  Dually,  a\q 
is  found  by  taking  taking  the  disjunction  of  the  set  of  formulae  each  of  which  is  a  conjunction 
of  a  set  of  pi  drawn  from  the  “left  hand  side”  of  the  dynamic  axioms  of  Q  (pf  D  [a]?,-)  such 
that  that  conjunction  of  the  corresponding  qt 's  implies  q. 

Consider  the  following  sample  axioms: 

A  D  [o](£  V  C) 

G  D  [a\B 
(F  A  E)D  \a]D 

In  this  case,  a\(C  V  D)  =  (A  A  G)  V  (F  A  E).  The  reason  for  this  is  that  ( B  V  C) 
conjoined  with  ->  B  implies  (C  V  D),  so  the  conjunction  of  the  corresponding  left-hand  sides 
(A  A  (?)  is  one  disjunct  of  a\(C  V  D).  Likewise,  the  formula  D  alone  implies  (C  V  D),  making 
the  corresponding  left-hand  side  ( F  A  E)  the  second  disjunct.  These  two  cases  exhaust  the 
possibilities  for  getting  ( C  V  D). 

The  reason  why  the  formulas  p/a  and  a\q  defined  in  this  way  are  not  exactly  equiv¬ 
alent  to  strongest  postcondition  and  weakest  precondition  lies  in  the  nature  of  our  atomic 
actions.  Briefly,  in  the  context  of  programming  languages  one  typically  begins  with  primitives 
whose  semantics  are  fully  characterized  and  focuses  on  characterizing  the  derived  operations 
(sequencing,  etc.)  [Dij75]  For  example,  the  weakest  precondition  for  the  assignment  primitive 
is  given  by  wp(“x  :==»  E",P{x))  =■*  P(E).  This  equation  asserts  that  the  weakest  precondition 
for  condition  P  and  action  “x  gets  E”  is  precisely  P  with  E  substituted  for  x. 

In  our  case,  however,  the  primitive  actions  are  specified  only  by  axioms  giving  one¬ 
way  implications.  Thus,  unless  we  make  assumptions  of  a  “non-monotonic”  nature,  we  would 
generally  be  able  to  consistently  add  axioms  that  “weaken”  the  precondition  or  “strengthen" 
the  postcondition  of  an  action.  Since  “provably  weakest”  is  unattainable,  we  make  do  with 


“weakest  provable.”  This  does  not  affect  the  completeness  of  the  search  algorithm,  since  we 
are  only  looking  for  programs  which  provably  satisfy  the  specifications. 

2.3.3  Hierarchical  Planning 

The  key  observation  in  extending  the  single-level  algorithm  to  multi-level,  hierarchical 
planning  is  that  an  atomic  action  at  level  k  is  a  plan  to  be  solved  for  at  level  k  +  1.  This  point 
of  view  is  possible  because  of  the  way  the  planning  problem  was  formalised.  Specifically,  an 
atomic  action  is  described  by  a  set  of  dynamic  axioms  in  Q.  Likewise,  the  desired  program 
is  described  by  a  set  of  dynamic  axioms  in  R.  Since  the  same  formal  objects,  namely  sets  of 
dynamic  axioms,  are  involved  in  both  cases,  it  is  natural  to  assume  as  primitive  some  action 
with  given  properties  at  level  k  and  then  solve  for  a  program  having  those  properties  at  level 
k  +  1. 

Formally,  a  hierarchical  planning  problem  is  a  tree  of  single-level  problems.  If  <V* 
<Pk<Ak>,Qk,Rk(uk)>  is  the  problem  at  non-leaf  node  k,  then  node  k  has  one  successor  for 
each  a*,;  in  Ak,  and  that  successor's  problem  has  the  form  <V*+i,Q*+1,<2/(afci,-)>  where  Q' 
denotes  the  subset  of  dynamic  axioms  of  Q*  having  the  form  p  D  [a*,s]9-  la  other  words,  the 
domain  constraints  on  the  primitive  “a”  at  level  k  become  plan  requirements  at  level  k+  1.  A 
solution  is  a  plan  using  the  vocabulary  of  the  leaf  nodes  that  satisfies  the  requirements  of  the 
root  node.  I.e.,  a  is  a  solution  if  it  solves  <ln,Qn,i?|(ui)>.  The  propositional  vocabulary 
and  action  vocabulary  can  change  from  level  to  level,  provided  the  domain  axioms  have  enough 
inferential  structure  to  make  the  transfer  from  level  to  level  meaningful. 

Obviously,  for  any  node  k,  only  the  successor  nodes  corresponding  to  actions  actually 
used  in  the  solution  need  be  solved.  Furthermore,  the  existence  of  a  solution  for  each  of  these 
nodes  guarantees  the  existence  of  an  overall  solution. 

As  with  other  hierarchic  planners,  the  main  benefit  of  levels  in  our  approach  is 
heuristic:  The  choice  of  intermediate  vocabularies  and  domain  axioms  constitutes  a  choice 


of  “planning  islands.”  Any  algorithm  that  tries  to  solve  a  problem  by  solving  the  nodes  in 
the  hierearchy  is,  in  essence,  searching  for  a  plan  constrained  to  go  through  the  states  defined 
by  the  intermediate  actions’  domain  constraints.  The  main  benefit  of  logic  here  is  to  define  a 
reasonable  relation  between  the  levels,  namely  the  relation:  “correctly  implements.” 

For  a  fixed  determination  of  levels  and  a  small  number  of  actions  it  would  be  possible 
to  precompute  solutions  to  the  subprobtems,  in  which  case  after  solving  the  problem  at  the  top 
level,  the  system  would  act  more  like  a  compiler  than  a  problem  solver.  In  dynamic  situations 
when  the  lower-level  actions  (in  effect,  the  “tools”  for  solving  the  problem)  are  changing  or 
when  only  a  small  number  of  actions  are  ever  actually  used,  it  seems  more  natural  to  solve 
subproblems  as  they  arise. 


2.4.  Discussion 


2.4.1  Modeling  Actions:  The  Legacy  of  STRIPS  Operators 

Much  of  the  research  into  the  control  of  planning  has  been  carried  out  in  the  STRIPS 
paradigm.  [Fik71,  Nil80]  In  this  approach,  actions  are  regarded  not  as  mappings  from  states 
to  states,  but  rather  as  syntactic  transformations  of  state-descriptions  to  state-descriptions, 
where  state-descriptions  are  logical  formulae.  One  consequence  is  the  oft-cited  benefit  of  not 
needing  to  mention  the  various  “frame  conditions,”  i.e.  the  properties  which  are  invariant 
under  an  action.8  Unfortunately,  the  need  for  operators  to  be  sensitive  to  the  syntax  of  state 
descriptions  led  researchers  to  consider  only  very  simple  state  descriptions  (e.g.  sets  of  atomic 
propositions)  and  very  simple  transformations  (e.g.  add-lists  and  delete-lists). 

however,  these  invariants  need  not  be  as  large  an  obstacle  to  practical  implementation  as  is  commonly  supposed 
(see  (Ros8l|). 
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As  an  example  of  an  action  that  is  difficult  to  specify  with  a  single  add-list/delete-list 
pair,  consider  the  action  toggle  described  by  a  pair  of  dynamic  axioms: 

On(liqht)  D  [toggle(awiteh)l->  On(light) 

->On(light)  D  \toggle(aviteh)]On(light) 

Since  the  post-condition  depends  conditionally  on  the  pre-condition,  it  cannot  be  determined 
in  isolation  whether  toggle  adds  or  deletes  the  wff  On(light).  The  same  would  hold  true  for 
actions  with  disjunctive  post-conditions. 

These  possibilities  notwithstanding,  many  planning  systems  do  make  the  assumption 
that  the  truth  of  a  given  atomic  proposition  in  the  state  resulting  from  applying  a  sequence 
of  operators  is  a  determinate,  calculable  thing.  Techniques  which  rely  crucially  on  these 
assumptions  are  sometimes  difficult  to  adapt  to  less  constraining  assumptions.  We  give  two 
illustrations  from  NOAH.  [Sac75] 

2.4.2  Nonlinear  Planning:  Problems  with  Partial  Orders  and  Shuffles 

The  basic  idea  behind  nonlinear  planning  is  the  following:  To  solve  a  conjunctive 
goal  G1&G2,  find  a  sequence  51  «  o;6;...;c  which  achieves  G 1  and  another  sequence  52  » 
d;  e; . . . ;  /  which  achieves  G 2.  Represent  the  overall  plan  as  a  network  of  partially  ordered 
actions  with  51  and  52  as  parallel  branches.  Now  use  the  “resolve  conflicts  critic”  to  detect 
interference  between  the  plans  and  impose  additional  ordering  constraints  among  the  actions 
to  rule  out  the  interference.  The  network  encodes  the  subset  of  possible  shuffles  of  51  with  52 
which  are  believed  to  achieve  the  overall  goal  G1&G2. 

For  the  resolve  conflicts  critic  to  filter  interference  correctly,  it  must  know  what  is 
true  at  each  node  of  the  network.  Unfortunately,  for  nodes  that  occur  after  joins,  what  is  true 
depends  crucially  on  the  ultimate  linerrization  of  the  parallel  branches.  In  the  general  case, 
the  best  that  c'  n  be  done  is  to  represent  the  disjunction  of  the  strongest  postconditions  of  the 


alternative  linearizations.7  This  requires  considering  the  alternatives,  of  which  there  are  (m^n) 
where  m  and  n  represent  the  lengths  of  the  action  sequences  in  the  two  parallel  branches.  Since 
it  is  easy  to  imagine  cases  where  resolve-conflicts  criticism  would  be  an  expensive  operation, 
the  belief  that  using  a  nonlinear  strategy  is  computationally  efficient  seems  to  be  grounded  in 
the  empirical  hypothesis  that  operators  encountered  in  practice  will  permit  easy  detection  of 
conflicts. 

2.4.3  Hierarchical  Planning:  Problems  with  Heuristic  Decompositions 

The  justification  for  partial  orderings  in  NOAH  is  tied  up  with  a  desire  not  to 
prematurely  commit  the  system  to  a  particular  linear  order  of  actions  which,  though  seemingly 
correct  at  one  level,  may  expand  into  incorrect  plans  at  lower  levels.  This  possibility  can  only 
arise,  of  course,  if  the  relation  between  levels  (“plan  A  achieves  the  same  effect  as  action  a”)  is 
not  exact.  However,  such  inexactness  undermines  the  original  rationale  for  hierarchic  planning, 
namely  factorization  of  complexity,  since  it  destroys  compositionality  and  requires  that  we 
check  complex  lower-level  plans  for  “unexpected”  global  interactions.  Again,  an  empirical 
hypothesis  is  presumably  invoked,  namely  that  by  some  suitable  metric,  the  plan  comes  “close” 
to  implementing  the  abstract  action.  (It  is  not  immediately  obvious,  though,  what  metric  could 
be  meaningful  for  the  space  in  question.) 

2.4.4  Some  Benefits  of  Bigression 

Some  of  the  benefits  of  regression  were  first  discussed  by  Waidinger  [Wal75]  and 
appreciated  by  Warren  [War74].  These  benefits  are  reaped  dually  by  including  progression, 
which  completes  the  logical  symmetry  and  allows  bidirectional  search.  As  we  have  described 

7 Actually,  NOAH  does  not  represent  disjunctive  postconditions  —  which  may  explain  why  disjunctive  goals  are 
considered  problematical. 


them,  the  progression  and  regression  operations  handle  arbitrary  boolean  formulas,  thus  solv¬ 
ing  conjunctive  and  disjunctive  goals  as  special  cases  of  a  more  general  strategy.  Goals  of 
maintenance  and  prevention  can  be  incorporated  into  the  algorithm  as  well  by  expressing  as 
(nonmodal)  wffs  the  condition  to  be  maintained  (m)  and  the  condition  to  be  prevented  (v). 
Since  the  planning  algorithm  actually  develops  a  descriptive  wff  ( d )  for  each  state  reachable 
during  plan  execution,  it  is  straightforward  to  add  a  check  to  the  procedures  LiveForward, 
LiveBackward,  and  NonTYiv  eliminating  paths  through  states  where  Q  | —  d  D  ->m  y  t/.8 
This  simple  approach  will  work  in  situations  where  no  dynamic  replanning  is  anticipated; 
goals  of  maintenance  and  prevention  involving  execution  monitoring,  feedback  and  replanning, 
require  more  complex  strategies. 
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3.  A  First-Order  Formalization  of  Knowledge 
and  Action  for  a  Multiagent  Planning  System 


This  section  was  written  by  Kurt  Konolige. 

3.1.  Introduction 

We  are  interested  in  constructing  a  computer  agent  whose  behavior  will  be  intelligent 
enough  to  perform  cooperative  tasks  involving  other  agents  like  itself.  The  construction  of 
such  agents  has  been  a  major  goal  of  artificial  intelligence  research.  One  of  the  key  tasks 
such  an  agent  must  perform  is  to  form  plans  to  carry  out  its  intentions  in  a  complex  world 
in  which  other  planning  agents  also  exist.  To  construct  such  agents,  it  will  be  necessary  to 
address  a  number  of  issues  that  concern  the  interaction  of  knowledge,  actions,  and  planning. 
Briefly  stated,  an  agent  at  planning  time  must  take  into  account  what  his  future  states  of 
knowledge  will  be  if  he  is  to  form  plans  that  he  can  execute',  and  if  he  must  incorporate  the 
plans  of  other  agents  into  his  own,  then  he  must  also  be  able  to  reason  about  the  knowledge 
and  plans  of  other  agents  in  an  appropriate  way.  These  ideas  have  been  explored  by  several 
researchers,  especially  McCarthy  and  Hayes  [McC69j  and  Moore  [M008O]. 

Despite  the  importance  of  this  problem,  there  has  not  been  a  great  deal  of  work  in 
the  area  of  formalizing  a  solution.  Formalisms  for  both  action  and  knowledge  separately 
have  been  examined  in  some  depth,  but  there  have  been  few  attempts  at  a  synthesis.  The 
exception  to  this  is  Moore’s  thesis  on  reasoning  about  knowledge  and  action  [M008O],  for 
which  a  planner  has  been  recently  proposed  [App80].  Moore  shows  how  a  formalism  based 
on  possible-world  semantics  can  be  used  to  reason  about  the  interaction  of  knowledge  and 
action.  In  this  paper  we  develop  an  alternative  formalism  for  reasoning  about  knowledge, 
belief,  and  action;  we  show  how  this  formalism  can  be  used  to  deal  with  several  well-known 
problems,  and  then  describe  how  it  could  be  used  by  a  plan  constructing  system. 
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3.1.1  Overview  and  Related  Work 

We  seek  a  formalization  of  knowing  and  acting  such  that  a  description  of  their  interac¬ 
tion  satisfies  our  intuitions.  In  the  first  section,  we  present  a  basic  formalism  for  describing 
an  agent’s  static  beliefs  about  the  world.  We  take  a  syntactic  approach  here:  an  agent’s 
beliefs  are  identified  with  formulas  in  a  first-order  language,  called  the  object  language 
(OL).  Propositional  attitudes  such  as  knowing  and  wanting  are  modeled  as  a  relation  be¬ 
tween  an  agent  and  a  formula  in  the  OL.  By  introducing  a  language  (the  metalanguage, 
or  ML)  whose  prime  object  of  study  is  the  OL,  we  are  able  to  describe  an  agent’s  beliefs 
as  a  set  of  formulas  in  the  OL,  and  express  partial  knowledge  of  that  theory.  An  agent's 
reasoning  process  can  be  modeled  as  an  inference  procedure  in  the  OL:  from  a  base  set 
of  facts  and  rules  about  the  world,  he  derives  a  full  set  of  beliefs,  called  his  theory  of  the 
world. 

The  syntactic  approach  to  representing  propositional  attitudes  is  well-known  in  the 
philosophy  literature,  and  in  the  artificial  intelligence  field  McCarthy  [McC79j  has  developed 
a  closely  related  approach.  The  formalism  developed  here  differs  mainly  in  that  it  explicitly 
identifies  propositional  attitudes  as  relations  on  sentences  in  an  object  language,  and  uses 
provability  in  the  OL  as  the  model  of  an  agent’s  reasoning  process.  We  are  able  to  present 
quite  complex  deductions  involving  the  beliefs  of  agents  (see  the  Wise  Man  Puttie  in 
Appendix  A,  for  example)  by  exploiting  the  technique  of  semantic  attachment  to  model 
directly  an  agent’s  reasoning  process.  We  are  indebted  to  Weyhrauch  [Wey80]  for  an 
introduction  to  this  technique,  and  for  the  general  idea  of  using  ML/OL  structures  to 
represent  agents. 

Finally,  our  work  differs  from  McCarthy's  in  its  careful  axiomatization  of  the  relation 
between  ML  and  OL,  and  incorporates  solutions  to  several  technical  problems,  including 
reasoning  about  belief- nesting  (beliefs  about  beliefs;  Creary  [Cre80]  has  also  described  a 
solution),  and  a  cleaner  approach  to  representing  quantified  OL  expressions  in  the  ML. 
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(This  latter  subject  is  not  directly  relevant  to  this  paper,  and  will  be  reported  in  [Kon81j.) 

An  alternative  to  the  syntactic  approach  to  representing  propositional  attitudes  is  the 
possible-world  approach,  so-called  because  it  utilizes  Kripke-type  possible-world  seman¬ 
tics  for  a  modal  logic  of  knowledge  and  belief.  Moore  [M008O]  has  shown  how  to  reason 
efficiently  about  propositional  attitudes  by  using  a  first-order  axiomatization  of  the  possible- 
world  semantics  for  a  modal  logic.  Our  objections  to  the  possible-world  approach  are 
twofold:  first,  the  possible-world  semantics  for  representing  propositional  attitudes  is  com¬ 
plex  and  at  times  unintuitive;  to  deduce  facts  about  an  agent’s  knowledge,  one  must  talk 
about  the  possible-worlds  that  are  compatible  with  what  the  agent  knows.  Ultimately,  we 
suspect  that  the  syntactic  approach  will  prove  to  be  a  simpler  system  in  which  to  perform 
automatic  deduction,  but  further  research  in  both  areas  is  needed  to  decide  this  issue.  A 
second  objection  is  that  it  seems  to  be  difficult  to  modify  possible-world  semantics  for  the 
modal  logic  to  model  adequately  inference  processes  other  than  logical  deduction.  The 
possible-world  approach  uses  the  modal  axiom  that  every  agent  knows  the  consequences 
of  his  knowledge,  and  this  is  obviously  not  true,  if  only  because  real  agents  have  resource 
limitations  on  their  reasoning  processes.  The  syntactic  approach  does  not  suffer  from  this 
criticism,  because  it  is  possible  to  describe  explicitly  in  the  ML  the  inference  procedure  an 
agent  might  use. 

The  second  part  of  this  paper  integrates  the  syntactic  approach  to  representing  knowledge 
and  belief  with  a  situation  calculus  [McC69]  description  of  actions.  We  concentrate  on  many 
of  the  interactions  between  knowledge  and  action  presented  in  Moore’s  thesis  [M008O]. 
Simply  stated,  Moore’s  account  is  that  an  agent’s  beliefs  in  any  situation  arise  from  at 
least  three  sources:  direct  observation  of  the  world,  persistence  of  beliefs  about  previous 
situations,  and  beliefs  about  what  events  led  to  the  current  situation.  By  formalizing  this 
assumption,  he  shows  how  to  model  in  an  intuitively  plausible  way  the  knowledge  an  agent 
needs  to  perform  actions,  and  the  knowledge  that  he  gains  in  performing  them.  Although 


we  subscribe  to  his  notions  on  how  knowledge  and  action  should  interact,  for  the  reasons 
stated  above  we  feel  that  the  possible-world  approach  Moore  uses  to  formalise  these  ideas, 
while  elegant,  may  not  have  the  same  intuitive  appeal  as  the  syntactic  approach. 

The  main  contribution  of  this  paper  is  to  show  that  the  syntactic  approach,  when 
integrated  with  a  situational  calculus  description  of  actions,  can  adequately  formalise 
Moore’s  criteria  for  the  interaction  of  knowledge  and  belief.  An  important  benchmark 
is  to  formalise  the  idea  of  a  test:  an  agent  can  perform  an  action  and  observe  the  result  to 
figure  out  the  state  of  some  unobservable  property  of  the  world.  We  conclude  the  second 
section  with  just  such  an  example. 

In  the  final  section  we  consider  the  application  of  these  results  to  a  planning  system,  in 
particular  one  that  would  require  an  agent  to  take  account  of  other  agents’  plans  in  forming 
his  own.  We  come  to  the  conclusion  that  such  a  planning  system  may  not  be  significantly 
different  from  current  situation  calculus  planners  in  its  method  of  search,  but  does  require 
considerably  more  sophistication  in  the  deductions  it  performs  at  each  node  in  that  search. 

3.2.  Agents’  Beliefs  and  First-Order  Theories 

In  this  section  we  lay  the  basic  groundwork  for  our  syntactic  approach  to  representing 
and  reasoning  about  agents’  beliefs.  We  will  model  an  agent’s  beliefs  about  the  world  as 
a  set  of  statements  (or  theory)  in  some  first-order  language  with  equality.  This  is  not  to 
say  that  an  agent  actually  represents  the  world  as  a  set  of  first-order  statements;  we  are 
not  concerned  here  with  the  details  of  the  internal  representation  of  a  computer  or  human 
agent  with  respect  to  its  environment.  All  we  seek  is  a  way  of  modelling  the  beliefs  of  an 
agent  in  a  manner  that  will  make  reasonable  predictions  about  the  agent’s  behavior,  and 
still  be  formally  tractable.  To  this  end  we  assume  that  we  can  represent  an  agent’s  beliefs 
about  the  world  as  a  set  of  statements  in  a  first-order  language,  and  model  the  derivation 
of  new  beliefs  by  an  agent  as  an  inference  process  on  those  statements. 


Consider  an  example  from  the  blocks-world  domain;  let  Ao  be  the  name  of  an  agent. 
Ao  will  have  some  set  of  beliefs  about  the  state  of  the  blocks-world.  We  represent  ^4o’s 
beliefs  as  a  list  of  well-formed  formulas  (wffs)  in  a  first-order  language  with  equality.  We 
call  this  list  of  wffs  .4o’s  theory  of  the  world.  For  example,  suppose  Ao  believes  that  block 
B  is  on  block  C,  and  that  he  is  holding  block  D.  Then  we  would  have: 

Ap's  Theory  of  the  Blocks- World 

ON{B,C) 

HOLDING(Ao,D) 

where  ON  and  HOLDING  have  the  appropriate  interpretations. 

Besides  specific  facts  about  the  state  of  the  world,  Ao  also  has  some  general  rules  about 
the  way  the  world  is  put  -cgether.  For  instance,  Ao  may  know  the  rule  that  if  any  block 
x  is  on  any  block  y,  then  the  top  of  y  is  not  clear.  Using  this  rule  together  with  specific 
beliefs  about  the  world,  he  may  be  able  to  deduce  that  C  is  not  clear.  This  can  be  modeled 
as  a  process  of  extending  Ao's  initial  set  of  beliefs  about  the  world  to  include  the  deduced 
information: 

/to ’s  Facts  and  Rules  about  the  World  ^  /Ip’s  Theory  of  the  World 

ON(B,C)  ON(B,C) 

HOLDING(Ao,  D)  HOLDING(Ao ,  D) 

VxyON(x,  y)D-CLEAR(y)  Vary  ON(x,  y)D~CLEAR(y) 

~CLEAR(C) 

Thus  an  agent's  theory  of  the  world  will  be  the  closure  of  a  set  of  facts  and  rules  about 
the  world,  under  some  suitably  defined  inference  procedure.  We  will  call  the  set  of  basic 
facts  and  rules  from  which  all  other  beliefs  are  derived  the  base  set  of  the  theory.  Note  that 
the  inference  procedure  that  derives  the  consequences  of  the  base  set  need  not  be  logical 


deduction;  it  is  readily  demonstrated  that  people  do  not  know  all  the  consequences  of  their 
beliefs,  that  they  derive  contradictory  consequences,  etc.  We  recognize  that  the  problem 
of  deriving  the  consequences  of  beliefs  for  more  realistic  inference  procedures  is  a  thorny 
and  unsolved  one,  and  do  not  intend  to  pursue  it  here.  For  the  purposes  of  this  paper  we 
have  chosen  logical  deduction  as  the  inferential  procedure:  an  agent  will  be  able  to  deduce 
the  logical  consequences  of  his  beliefs. 

3.2.1  Metalanguage  and  Object  Language 

If  we  were  always  to  have  complete  knowledge  of  an  agent’s  beliefs,  then  it  would  be 
possible  to  use  a  simple  list  of  facts  and  rules  to  represent  the  base  set  of  those  beliefs. 
However,  it  is  often  the  case  that  our  knowledge  is  incomplete;  we  may  know  that  an 
agent  either  believes  fact  P  or  fact  Q,  but  we  don’t  know  which.  Such  a  description  of 
an  agent’s  beliefs  cannot  be  modeled  by  a  list  of  facts.  So  the  modelling  process  must 
be  extended  to  a  description  of  an  agent’s  beliefs.  Since  beliefs  are  wffs  in  a  first-order 
language,  a  metalanguage  can  be  used  to  describe  a  collection  of  such  wffs  [Kle67j.  The 
basic  idea  is  to  have  terms  in  the  metalanguage  to  denote  syntactic  expressions  in  the 
first-order  language  used  to  encode  an  agent’s  beliefs.  The  latter  first-order  language  is 
called  the  object  language,  or  OL,  since  it  is  the  object  of  study  of  the  metalanguage  (ML). 
Predicates  in  the  metalanguage  are  used  to  state  that  an  expression  of  the  object  language 
is  in  an  agent’s  theory  of  the  world.  The  full  expressive  power  of  the  metalanguage  is 
available  for  describing  a  given  theory  of  the  object  language. 

It  is  natural  to  choose  a  first-order  language  for  the  metalanguage,  since  we  will  be 
interested  in  proof  procedures  in  the  ML  as  well  as  the  OL.  Let  ML  be  a  sorted,  first-order 
language  with  variables  restricted  to  range  over  particular  sorts.  The  domain  of  discourse  of 
the  ML  will  be  both  the  syntactic  expressions  of  the  ML,  as  well  as  the  domain  of  discourse 
of  the  OL.  Thus  the  ML  will  be  able  to  state  relationships  that  hold  between  OL  expressions 
and  the  actual  state  of  the  world. 
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A  basic  division  of  sorts  of  the  ML  is  between  terms  that  denote  individuals  in  the 
world,  and  terms  that  denote  expressions  in  the  OL.  Among  the  former  will  be  terms  that 
denote  agents  (Ao,Ai,. . .)  and  agents’  theories  of  the  world;  these  will  be  called  7/  terms. 

We  will  use  the  function  th  of  one  argument,  an  agent,  to  denote  that  agent’s  theory  of  the 
world. 

The  other  major  sort  of  terms  will  denote  formulas  of  the  OL;  these  will  be  referred 
to  as  Tf  terms.  Restricting  our  attention  for  the  moment  to  sentential  formulas  of  OL, 
there  will  be  terms  in  ML  that  denote  propositional  letters  in  OL,  and  constructors  in  ML 
for  putting  together  more  complicated  formulas  from  these  letters.  For  example,  P'  in  ML 
denotes  the  propositional  letter  P  of  the  OL,1  and  the  ML  term  and(P',Q')  denotes  the 
sentence  PaQ  of  the  OL.  These  ML  constructors  form  an  abstract  syntax  [McC62]  for  OL 
expressions. 

Writing  names  of  formulas  using  and,  or,  not,  and  imp  as  constructors  is  somewhat 
cumbersome.  For  the  most  part  we  will  use  a  syntactic  abbreviation,  enclosing  an  OL 
formula  in  sense  quotes,2  to  indicate  that  the  standard  ML  term  for  that  formula  is 
intended.  For  example,  we  will  write: 

rPACp  for  and(P>,  Q') 

rPD(gvi2p  for  im^P*  ,or{Q'  ,R')) 

and  so  on 

The  rule  for  translating  sense-quote  abbreviations  into  Tf  terms  of  the  ML  is  to  replace 
each  predicate  symbol  P  of  the  sense-quote  expression  by  the  ML  term  symbol  P' ,  and 
each  boolean  connective  by  the  corresponding  ML  boolean  constructor.  As  more  sorts  are 
introduced  into  the  ML  we  will  extend  the  sense-quote  convention  in  various  ways. 

Finally,  we  introduce  the  ML  predicates  TRUE,  FACT,  and  PR,  each  of  which  has 

1  The  general  convention  will  be  to  use  primed  terms  in  ML  to  denote  the  corresponding  unprimed  formulas  in  OL. 

2 They  are  called  sense-quotes  to  indicate  that  the  sense  of  the  expression  is  wanted,  rather  than  its  truth-value.  In 
(Kap71]  these  are  called  Frege  quotes. 
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an  OL  formula  as  one  of  its  arguments.  TRUE(f),  where  /  is  an  OL  formula,  means  that 
/  is  actually  true  in  the  world  under  consideration.  It  is  often  the  case  that  we  will  want 
to  describe  a  certain  condition  actually  holding  in  the  world,  independent  of  whether  some 
agent  believes  it  or  not;  for  instance,  this  is  critical  to  our  reasoning  about  events  in  the 
next  section,  where  events  are  defined  as  transformations  from  one  state  of  the  world  to 
another. 

We  intend  TRU E  to  have  the  normal  Tarskian  definition  of  truth,  so  that  the  truth- 
recursion  axioms  are  valid.  Let  the  variables  /  and  g  range  over  OL  expressions.  Then  we 
can  write  the  metalanguage  axioms  for  truth-recursion  in  the  object  language  as  follows: 


V/  ~TRUE{f)  =  TRUE(not(f)) 
Vfg  TRUE(f)vTRUE{g)  ==  TRUE(or(f,g)) 
Vfg  TRUE{f)ATRUE{g)  =  TRUE(and(f,g)) 
Vfg  TRUE(f)DTRUE(g)  s  TRUE{imp(f,g)) 


(TR) 


FACT(t,f),  where  t  is  an  OL  theory,  means  that  /  is  one  of  the  base  set  formulas 
of  the  theory  (and  from  which  the  rest  of  the  theory  will  be  derived  by  deduction).  Using 
FACT,  agent  ylo’s  previously  exhibited  beliefs  about  the  world  could  be  described  by  the 
following  ML  predicates: 

FACT(th(Ao),  rON(B,C f) 

FACT(th(Ao),  rHOLDING(A0,D)]) 

FACT(th(Ao),  rVxV  ON{z,v)D~CLEAR(vV) 

The  last  FACT  predicate  describes  a  rule  that  agent  Ao  believes. 

One  special  type  of  FACT  that  we  will  make  frequent  use  of  is  a  formula  known  to  all 


agents.  We  define  the  predicate  CFACT  on  OL  expressions  to  mean  that  a  true  expression 
is  a  FACT  for  all  agents,  that  is,  a  Common  FACT: 


VfCFACT(f)  D  Va  FACT(th{a),  f)ATRUE(f) 


(CF 1) 


CF  doesn't  completely  axiomatize  what  we  intend  a  common  fact  to  be,  however,  since 
it  doesn’t  say  that  every  agent  knows  that  every  agent  knows  that  every  agent  knows  /, 
etc.3  But  a  fuller  characterization  of  CFACT  must  wait  until  the  technical  machinery  for 
describing  belief-nesting  is  developed  in  a  later  subsection. 

PR(t,f)  means  that  /  is  provable  in  the  theory  t.  As  discussed  previously,  we  will 
assume  that  PR  gives  the  closure  of  sentences  in  OL  that  can  be  generated  by  logical 
deduction  from  an  original  set  of  FACTs.  A  simple  axiomatization  of  PR  can  be  given  for 
Hilbert-style  (assumption-free)  proofs.  There  is  only  one  rule  of  inference,  Modus  Poaeas: 

Vtfg  PR(t,  imp(f,  g))APR(t,  f)  D  PR(t,  g)  ( MP ) 

that  is,  from  PDQ  and  P  in  the  OL,  infer  Q.  Since  every  FACT  is  an  initial  theorem  of 
the  theory,  we  assert  that  each  of  these  is  provable: 

Vt/  FACT(t,  f)DPR(t,  /).  (FP) 

And  in  each  theory  the  logical  axioms  of  a  Hilbert  system  need  to  be  asserted;  we  assume 
a  sufficient  set  for  the  sentential  case. 

MP  and  the  Hilbert  axioms  will  be  used  in  ML  proofs  of  the  provability  of  OL 
statements;  these  axioms  simulate  a  Hilbert-type  proof  system  for  an  OL  theory.  This 
simulation  is  necessary  because  in  general  there  will  be  an  incomplete  ML  description  of 
the  OL  theory,  rather  than  a  simple  list  of  FACTs  for  that  theory.  In  those  special  cases 
when  a  list  of  FACTs  is  available,  it  is  possible  to  run  the  proof  procedure  on  the  OL 
theory  directly.  That  is,  since  the  intended  meaning  of  the  PR  predicate  is  provability 
in  the  OL  theory,  we  can  check  whether  the  PR  predicate  holds  in  the  ML  by  running 
a  theorem- prover  in  the  OL.  It  also  isn’t  necessary  to  use  a  Hilbert  system,  and  we  will 


feel  free  to  exploit  any  system  of  natural  deduction  that  is  sound.  The  technique  of  using 
a  computable  model  of  the  intended  interpretation  of  a  predicate  to  determine  the  truth 
of  formulas  involving  that  predicate  is  called  semantic  attachment  [Wey80j,  and  it  will  be 
used  extensively  to  simplify  proofs  in  later  sections. 

The  provability  predicate  PR  does  not  have  the  same  characteristics  as  TRUE,  and 
this  is  important  in  representing  beliefs.  For  example,  the  fact  that  P  is  not  provable  doesn't 
imply  that  ~P  is  provable.  If  we  identify  provability  with  belief,  ~PR(th(Ao),  rP1 )  asserts 
that  P  is  not  one  of  Ao’s  beliefs  about  the  world,  but  this  does  not  imply  PR(th(Ao),  !~~-'P1 ), 
i.e.,  that  Ao  believes  ~P.  Also,  it  is  possible  to  express  that  either  Ao  believes  that  C  is 
clear,  or  he  believes  that  C  is  not  clear: 

PR{th(Ao),  rCLEAR( Cf  )VPR(th(Ao ),  r~~CLEAR{C? ); 

this  says  something  quite  different  from  PR(th(Ac),  rCLEAR(C)V'~~'CLEAR(Cp);  the 
latter  is  a  tautology  that  every  agent  believes,  while  the  former  says  something  a  lot  stronger 
about  Ao' s  beliefs  about  the  world. 

Paralleling  the  truth  recursion  axioms  TR,  we  can  state  rules  for  the  provability  of 
compound  OL  expressions  in  terms  of  their  immediate  subexpressions.  Because  of  the 
nature  of  provability,  the  axioms  for  negation,  disjunction,  and  implication,  unlike  their 
truth-theoretic  counterparts,  are  not  equivalences. 


Vt/  — PR(t,  /)  C  PR(t,  not{f)) 
Vt/g  [Pfl(t,/)VPfl(*,*)I  D  PR(t,or(f,g)) 
Vt/g  [PR(t,f)APR(t,g)\  =  PR(t,and(f,g)) 
Wtfg\PR(t,f)DPR(t,g) J  C  PR(t,imp(f,g)) 


(PR) 


These  are  all  deducible  from  the  logical  axioms  in  the  Hilbert  proof  system;  for  instance, 
the  last  assertion  is  just  a  restatement  of  Modus  Ponens. 

Another  interesting  connection  between  the  PR  and  TRUE  predicates  can  be  drawn 
by  looking  at  models  of  the  OL.  Suppose  we  have  used  FACT  and  PR  to  describe  an 
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agent’s  theory  T  of  the  world.  There  will  be  some  set  of  models  that  satisfy  T,  i.e.,  for 
which  all  of  7"s  theorems  hold.  The  actual  world  will  be  one  of  these  models  just  in  case 
all  T's  theorems  hold  for  the  world.  This  condition  is  statable  in  the  ML  as: 


V/  PR(T,  f)DTRUE(f) 


In  general  this  assertion  will  not  be  valid,  that  is,  an  agent’s  beliefs  need  not  correspond  to 
the  actual  world.  By  introducing  the  predicate  TRUE  in  the  ML,  we  are  able  to  state  the 
correspondence  between  a  given  theory  of  the  world  and  the  actual  state  of  affairs  in  the 
world. 

3.2.2  Knowledge  and  Belief 

The  PR  and  TRUE  predicates  can  be  used  to  state  our  fundamental  definitions  of 
knowing  and  believing  for  an  agent.  BEL(a,  f)  means  that  agent  a  believes  /;  KNOW( a,  f) 
means  that  agent  a  knows  /.  Then  we  have  the  definitions: 


Va/  BEL(a,f)  =  PR{th(a),f) 

Va/  KNOW(a,f)  =  BEL(a,  f)ATRUE{f) 


(B 1) 


That  is,  we  identify  belief  with  provability  in  an  OL  theory,  and  knowledge  as  a  belief  that 
actually  holds  in  the  world.  In  model-theoretic  terms,  a  sentence  is  known  to  an  agent  if 
the  sentence  holds  in  all  of  his  models,  and  the  actual  world  is  a  model  for  that  sentence. 
The  definition  of  a  common  fact  in  CFl  means  that  all  common  facts  are  known  to  all 
agents. 

We  already  know  that  the  inference  process  used  in  deriving  new  beliefs  from  old  ones 
is  only  approximated  as  logical  consequence,  yet  we  should  still  expect  this  approximation 
to  correctly  model  some  of  the  characteristics  we  attribute  to  belief.  For  instance,  if  a 
rational  agent  believes  that  PDQ,  and  he  doesn’t  believe  Q,  then  it  should  be  the  case  that 
he  doesn’t  believe  P.  Translating  to  the  above  notation  yields  the  sentence: 
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BEL(Ao,  rPDQ1)A-BEL(A0,  rQn)  D  -BEl+M,  rP1) 


To  illustrate  the  use  of  axioms  for  belief  and  provability  given  so  far,  we  exhibit  a  natural 
deduction  proof  of  this  sentence  in  ML: 


1. 

BEL(Ao,  rPDCp) 

given 

2. 

PR(th(Ao),rPDQn) 

1,51 

3. 

~BEL(Ao,rQn) 

given 

4. 

~PR(th(Ao),  rQ1) 

3,51 

5. 

PR(th(A*),  rP1)DPR(tA(Ao),  rQn) 

2, PR 

6. 

~PR(th(M),  rPn) 

4,5  contrapositive 

7. 

— BEL(Ao,  rP1) 

6,51 

This  particular  proof  in  the  ML  cannot  be  done  by  semantic  attachment  to  the  OL,  because 
it  involves  reasoning  about  what  isn’t  provable  in  the  OL  theory. 

At  this  point  we  have  presented  the  basic  ideas  and  definitions  for  a  syntactic  approach 
to  representing  and  reasoning  about  agents’  beliefs.  The  rest  of  this  section  is  devoted  to 
exploring  various  technical  issues  that  arise  when  extending  the  previous  analysis  to  talking 
about  individuals. 

3.2.3  Individuals 

By  restricting  ourselves  to  the  case  of  sentential  formulas  in  OL,  we  have  been  able  to 
present  the  basic  concepts  for  representing  the  beliefs  of  an  agent  more  simply.  Additional 
complications  arise  when  dealing  with  terms  in  the  OL  that  denote  individuals  rather  than 
truth-values.  But  a  ML  encoding  of  these  terms  is  necessary  in  order  to  express  such 
concepts  as  agent  A0  knows  who  B  is. 

To  talk  about  the  individuals  that  the  OL  refers  to,  we  introduce  an  additional  sort 
into  the  ML,  whose  denotation  will  be  the  function  terms  of  the  OL.  This  sort  will  be  called 
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TV,  and  consists  of  the  following  members: 

(1)  variables  or,/?,...; 

(2)  ,*«)},  where  f»-  €  TV; 

(TT) 

(3)  » j(t),  where  t  E  Tj  [the  “standard  name”  function]; 

(4)  nothing  else. 

The  ML  variables  a,  0,  . . range  over  OL  function  terms.  For  example,  we  can  state  that 
Ao  believes  a  particular  block  is  on  C  by  asserting  the  ML  expression: 

3a  BEL^Ao,  ON'(a,  C')). 

In  this  expression  there  are  two  ML  terms  in  TV,  namely,  a  and  C#.  C  is  a  O-ary  function 
(or  constant)  in  TV  that  denotes  the  constant  term  C  in  OL.4  ON'  is  a  type  of  ML  term 
that  hasn’t  been  used  explicitly  before;  it  is  a  member  of  T>  because  it  names  an  OL 
formula.  It  takes  two  arguments,  each  of  which  is  a  ML  term  denoting  an  OL  term,  and 
constructs  an  OL  formula  that  is  the  OL  predicate  ON  of  these  arguments.  So  the  ML 
term  ON'(a,C')  denotes  the  OL  expression  ON(A,C),  where  A  is  the  OL  term  denoted 
by  a. 

It  is  now  possible  to  give  a  full  definition  of  Tp  terms: 

(1)  variables 

(2)  {/"(*!» ••  •»*»»)}»  where  f,-  €  TV  [boolean  constructors,  e.g.,  and]; 

{ TF ) 

(3)  { . . .,  <„)},  where  t,-  6  TV  [predicate  constructors,  e.g.,  ON']) 

(4)  nothing  else. 

and  7/  terms: 

*  We  extend  the  prime  convention  to  cover  ML  terms  in  Tt  as  well  as  Tp  ;  that  is,  t?  in  ML  denotes  the  unprimed 
term  t  in  OL. 


(1)  variables  x,y,...; 

(2)  {/"(<i»  •  •  •,  <n)},  where  *,•  6  7>  (individual  constants  and  functions]; 

(77) 

(3)  A (t),  where  t  €  TV  (the  denotation  function]; 

(4)  nothing  else. 

We  will  also  find  it  convenient  to  extend  the  notion  of  sense-quote  abbreviations  to 
handle  ML  terms  involving  7Y  variables.  The  previous  rules  are  expanded  in  the  following 
way:  all  function  symbols  in  the  sense-quote  expression  are  replaced  by  their  primed 
forms,  while  any  symbols  used  as  variables  in  the  surrounding  ML  expression  remain 
unchanged.  For  example,  the  sense-quote  expression  in  3a  K  NOW(Aq,  rOiV(a,6(C’)j1) 
is  to  be  understood  as  a  syntactic  abbreviation  for  the  ML  term  ON^a,  (/(C1)).  We  have 
not  yet  said  what  happens  to  7>  variables  in  sense-quote  expressions;  this  must  wait  until 
standard  names  are  explained  in  the  next  subsection. 

The  introduction  of  7r  terms  into  the  ML  completes  the  descriptive  power  of  ML 
for  OL  expressions.  It  also  lets  us  handle  some  of  the  well-known  denotational  puzzles 
in  the  philosophy  literature.  One  of  the  simplest  of  these  is  the  Morningstar-Eveningstar 
description  problem.  Both  Morningstar  and  Eveningstar  are  actually  the  planet  Venus 
seen  at  different  times  of  the  day.  An  agent  .4o  believes  that  they  are  not  the  same; 
further,  he  doesn’t  have  any  knowledge  about  either  being  the  planet  Venus.  Let  MS, 
ES,  and  VENUS  be  OL  terms  that  denote  the  Morningstar,  the  Eveningstar,  and  Venus, 
respectively.  The  following  set  of  ML  formulas  describes  this  situation: 

TRUE{rES  =-  VENUS**) 

TRUE(rMS  =  VENUS' 1) 

BE1\Aq,  rMS  ES1) 

~BEL(Ao,  rES  =*  VENUS**) 

~BEL(Ao,  rMS  —  VENUSn) 

It  is  perhaps  easiest  to  explain  this  set  of  sentences  in  model-theoretic  terms.  The  intended 
interpretation  of  the  the  OL  terms  ES,  MS,  and  VENUS  is  the  same  object,  namely 


the  planet  Venus.  The  two  TRUE  predicates  establish  this,  since  they  assert  that  these 
three  terms  denote  the  same  individual  in  the  world.  On  the  other  hand,  the  first  BEL 
predicate  asserts  that  in  the  models  of  Ao's  theory  of  the  world,  MS  and  ES  denote 
different  individuals.  This  means  that  the  actual  world  cannot  be  among  the  models  of  this 
theory.  Further,  the  last  two  BEL  predicates  assert  that  ES  and  MS  are  not  provably 
equal  to  VENUS  in  this  theory;  hence  there  will  be  some  models  of  the  theory  for  which 
ES  =  VENUS  holds,  some  for  which  MS  =  VENUS  holds,  and  some  for  which  neither 
holds.  From  this  we  conclude  that  not  only  is  Ao  mistaken  as  to  the  equality  of  ES  and 
MS,  he  also  is  unsure  about  whether  either  is  the  same  as  VENUS.  [McC79]  lists  some 
other  philosophical  puzzles  that  can  be  handled  in  a  syntactic  formulation. 

3.2.4  Knowing  Who  Someone  Is 

One  of  the  problems  that  any  formal  treatment  of  belief  must  confront  is  that  of 
describing  when  an  agent  knows  who  or  what  something  is.  For  example,  the  following  two 
English  sentences  say  something  very  different  about  the  state  of  .Ao’s  knowledge:5 

(1)  Mo  knows  who  murdered  John.” 

(2)  Mo  knows  that  someone  murdered  John.” 

The  police  would  certainly  be  interested  in  talking  to  Ao  if  the  first  statement  were  true, 
while  the  second  statement  just  means  that  Ao  read  the  local  tabloid.  We  might  paraphrase 
the  first  statement  by  saying  that  there  is  some  individual  who  murdered  John,  and  Ao 
knows  who  that  individual  is.  The  second  statement  can  be  true  without  Ao  having  any 
knowledge  about  the  particular  individual  involved  in  the  murder. 

How  is  the  distinction  between  the  two  sentences  above  to  be  realized  in  this  formalism? 
The  second  sentence  is  easy  to  represent: 

BEL(Ao,  r3x  MURDERED{x,  JOHN T )  ( Wl ) 

4  A  similar  problem  appears  in  [Qui71| 
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This  simply  says  that  Aq  believes  in  the  existence  of  an  individual  who  murdered  John.  It 
might  be  supposed  that  the  first  sentence  could  be  represented  in  the  following  way: 

3a  BEL(Ao,  rMURDERED(a,  JOHN? )  (W2) 

W2  says  that  there  is  a  MURDERED  predicate  in  Ao’s  theory  of  the  world  relating  some 
individual  (a’s  denotation)  and  John.  Unfortunately,  this  isn’t  quite  strong  enough;  if  the 
denotation  of  a  is  the  OL  term  murder cr(JOHN),  then  W2  is  virtually  a  tautology,  and 
doesn’t  say  that  Ao  knows  who  murdered  John.  Indeed,  if  the  OL  expression  in  W1  is 
skolemized,  it  becomes  obvious  that  W 1  and  W 2  are  equivalent. 

What  seems  to  be  going  on  here  is  that  different  names  have  a  different  status  as  far  as 
identifying  individuals  is  concerned.  “Bill’’  is  a  sufficient  description  for  identifying  John’s 
murderer,  whereas  “John’s  murderer”  is  not.  The  question  of  what  constitutes  a  sufficient 
description  is  still  being  debated  in  the  philosophical  literature.  But  for  the  purposes  of 
this  paper,  it  will  suffice  if  we  have  a  name  that  is  guaranteed  to  denote  the  same  individual 
in  every  model  of  the  OL.  By  asserting  a  predicate  involving  this  name  in  Ao’s  theory  of 
the  world,  it  will  be  possible  to  encode  the  fact  that  Ao  believes  that  predicate  for  the  given 
individual.  Names  that  always  denote  the  same  individual  are  called  standard  names. 

The  formal  method  of  establishing  standard  names  is  straightforward.  Consider  the 
set  of  all  individuals  involved  in  the  situation  we  wish  to  consider.9  Include  in  the  OL  a 
set  of  constant  symbols,  the  standard  name  symbols,  to  be  put  in  one-one  correspondence 
with  these  individuals.  The  language  OL  will  be  partially  interpreted  by  specifying  this 
correspondence  as  part  of  any  model  of  the  language;  this  means  that  the  only  models  of 
OL  we  will  consider  are  those  that  are  faithful  to  the  standard  name  mapping. 

In  the  metalanguage,  we  introduce  the  standard  name  function  tf  of  one  argument  (see 
the  definition  of  7Y  terms  above).  This  function  returns  the  standard  name  of  its  argument. 
Generally  we  will  use  lower  case  Greek  letters  from  the  later  part  of  the  alphabet  as  ML 


*  We  restrict  ourselves  to  countable  sets  here. 


variables  for  OL  standard  names  [ft,  v,  . . .].  The  metalanguage  statement  of  *Ao  knows 
who  the  murderer  of  John  is”  then  becomes: 

(17 (x)  =  n)  A  KNOW{Ao,  rM U RDERED(fi,  JOHN)1 )  ( W3) 

Because  /1  denotes  a  standard  name,  the  only  models  of  this  statement  are  those  in  which 
the  same  individual  x  murdered  John.  This  is  in  contrast  to  Wl  and  W2  above,  which 
allow  models  in  which  any  individual  murdered  John.  An  immediate  consequence  is  that 
W 1  and  W 2  are  derivable  from  W3,  but  not  the  other  way  around. 

So  in  order  to  assert  that  Ao  knows  who  or  what  some  individual  B  is,  we  write  in  the 
ML:7 

3xp  (V(x)  —  ft)  A  KNOW(Ao,  rB  —  /T1 ) 

By  modifying  the  sense-quote  translation  rules  slightly,  it  is  possible  to  write  OL  expressions 
involving  standard  names  much  more  compactly.  The  modification  is  to  assume  that  any 
ML  variable  of  type  T/  occurring  within  a  sense-quote  gets  translated  to  the  standard 
name  of  that  variable.  With  this  rule,  for  example,  the  above  assertion  comes  out  as 
3x  KNOW(Ao,  rB  —  xn ). 

We  will  use  the  predicate  KNOWIS(a,  0)  to  mean  that  the  agent  a  knows  who  or 
what  the  OL  term  denoted  by  0  refers  to.  The  definition  of  KNOWIS  is: 

Wa0  KNOWIS{a,0)ss3xKNOW(a,  r0  =  p)  [KW) 

Note  that  the  property  of  being  a  standard  name  is  a  relation  between  a  term  of  the  OL  and 
models  of  this  language,  and  hence  cannot  be  stated  in  the  OL.  The  use  of  a  metalanguage 
allows  us  to  talk  about  the  relation  between  the  OL  and  its  models. 


7  This  analysis  essentially  follows  that  of  [Kap71|,  with  the  extension  of  standard  names  to  all  individuals  in  the 
domain,  rather  than  just  numbers  and  a  few  other  abstract  objects.  There  are  problems  in  using  standard  names  for 
complex  individuals,  however;  see  )Kap7l]. 
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One  of  the  proof-theoretic  consequences  of  using  standard  names  is  that  every  theory 
can  be  augmented  with  inequalities  stating  the  uniqueness  of  individuals  named  by  standard 
names.  In  the  metalanguage,  we  write: 

Vxy  *  ^  V  D  Vt  PR(t,  rx^yn)  (SN) 

Formally,  the  definition  of  a  standard  name  can  be  axiomatized  in  the  ML  by  intro¬ 
ducing  the  denotation  function  A.8  A(a),  where  a  denotes  an  OL  term,  is  the  denotation 
of  a  in  the  actual  world;  it  is  the  inverse  of  the  standard  name  function,  since  it  maps  an 
OL  term  into  its  denotation.  There  is  an  intimate  relation  between  the  denotation  function 
and  equality  statements  in  OL  formulas  describing  the  world: 

V<*0  TRUE(  ro  —  (P )  a>  A(t»)  —  A(0)  (Z)l) 

that  is,  two  OL  terms  are  equal  in  the  actual  world  just  in  case  they  denote  the  same 
individual;  D\  can  be  viewed  as  a  definition  of  the  intended  interpretation  of  equality.  The 
prime  purpose  of  the  denotation  function  is  to  tie  together  the  denotation  of  terms  in  the 
OL  and  the  ML.  For  standard  names,  it  can  be  used  to  state  that  the  denotation  of  a 
standard  name  is  the  same  individual  in  all  situations,  something  that  cannot  be  done  with 
equality  predicates  in  the  OL: 

V*A(q(*))  — *  (D2) 

For  example,  by  asserting  ti(VENUS)  =  VENUS'  in  ML,  we  fix  the  denotation  of  the 
OL  term  VENUS'  to  be  the  individual  denoted  by  the  ML  term  VENUS  in  all  models 
of  the  OL. 

The  introduction  of  standard  names  with  fixed  denotations  across  all  models  makes 

•This  is  Church's  denotation  predicate  in  function  form  [Chu51|;  since  a  term  can  have  only  one  denotation,  it  is 
simpler  to  use  a  function. 
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the  task  of  relating  the  OL  to  the  ML  easier.  By  introducing  this  “common  coin”  for 
naming  individuals,  we  are  able  to  write  expressions  of  the  OL  that  represent  beliefs  without 
constantly  worrying  about  the  subtle  consequences  of  the  denotational  variance  of  terms  in 
those  expressions.  Standard  names  will  play  an  important  role  in  describing  belief-nesting 
(beliefs  about  beliefs),  in  describing  executable  actions,  and  in  simplifying  the  deduction 
process. 

3.2.5  The  Object  Language  as  Metalanguage 

In  this  subsection  we  extend  the  OL  to  include  a  description  of  another  object  language 
OL7.  Thus  extended,  the  OL  can  be  viewed  as  a  metalanguage  for  OL7.  The  reason  we 
want  to  do  this  is  that  it  will  be  necessary  for  representing  an  agent’s  view  of  a  world 
that  is  changing  under  the  influence  of  events.  In  the  next  section  we  will  show  how  an 
agent  can  model  the  way  in  which  the  world  changes  by  describing  what  is  true  about 
different  states  of  the  world  connected  by  events.  But  to  describe  these  states  of  the  world, 
or  situations,  the  agent’s  theory  must  talk  about  sentences  of  another  language  holding  in 
a  given  situation. 

Before  trying  to  extend  the  formal  apparatus  of  the  OL  to  describe  another  OL,  it 
is  helpful  to  examine  more  closely  the  relation  between  the  ML  as  a  means  of  studying 
the  OL  and  as  a  means  of  describing  the  actual  world.  This  is  because  the  structure  of  a 
ML/OL  pair  will  be  very  similar  no  matter  what  the  depth  of  embedding;  and  the  simplest 
such  structure  to  study  is  obviously  the  topmost  one.  Although  we  initially  characterized 
the  ML’s  domain  of  discourse  as  including  that  of  the  OL,  it  appears  that  we  have  not 
made  much  use  of  this  characterization.  In  describing  the  models  of  OL,  however,  it  was 
necessary  to  pick  out  the  model  that  was  the  actual  world;  this  was  done  with  the  predicate 
TRUE.  And  it  was  impossible  to  state  the  definition  of  a  standard  name  without  appealing 
to  terms  in  the  ML  that  referred  to  individuals  in  the  actual  world.  So,  in  fact,  we  have 
already  used  the  ML  to  characterize  the  actual  state  of  the  world  and  the  individuals  that 
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populate  it. 

We  have  stated  that  agents’  beliefs  are  represented  as  first-order  theories  of  the  world. 
The  ML  is,  by  the  above  argument,  just  such  a  theory;  but  whose  theory  of  the  world  is 
it?  One  useful  interpretation  is  to  take  what  we  will  call  the  egocentric  view:  a  theory 
in  the  ML  is  identified  as  the  theory  of  a  particular  agent.  That  is,  suppose  we  were  to 
build  a  computer  agent  and  invest  him  with  a  ML/OL  structure  as  a  way  of  representing 
other  agents’  beliefs.  Then  the  nonlogical  axioms  of  the  ML  would  constitute  the  computer 
agent’s  theory  of  the  world.  The  interpretation  of  the  ML  predicate  TRUE  would  be  “what 
the  computer  agent  believes  about  the  world,”  and  of  the  predicate  K NOW,  “what  another 
agent  believes  that  agrees  with  what  the  computer  agent  believes.”  In  this  interpretation, 
there  is  no  sense  of  absolute  truth  or  knowledge;  the  beliefs  of  one  agent  are  always  judged 
relative  to  those  of  another. 

Suppose  we  identify  the  agent  Aq  with  the  ML;  what  interpretation  does  the  OL 
theory  th(Ao)  have?  Interestingly  enough,  it  is  .do’s  introspective  description  of  his  own 
beliefs.  Unlike  other  agents'  theories  of  the  world,  th(Ao)  shares  an  intimate  connection 
with  formulas  that  hold  in  the  ML.  For  a  rational  agent,  it  should  be  the  case  that  if 
he  believes  P,  then  he  believes  that  he  believes  P.  We  can  state  this  connection  by  the 
following  rule  of  inference: 

Belief  attachment:  If  the  agent  a  is  identified  with  the  ML,  then  from 
TRUE(f)  infer  BEL(th(a),  /). 

Introspection  will  be  useful  when  we  consider  planning,  because  a  planning  agent  must  be 
able  to  reflect  on  the  future  state  of  his  beliefs  when  carrying  out  some  plan. 

If  the  metalanguage  is  intended  to  describe  the  actual  world,  then  it  is  reasonable  to 
ask  what  the  relation  is  between  models  of  the  ML  and  models  of  its  OL,  and  whether 
this  connection  can  be  formalized  in  the  ML.  We  start  by  adding  predicate  symbols  to  the 
ML  whose  intended  meaning  is  a  property  of  the  actual  world,  rather  than  of  the  OL  and 


its  models.  Consider  such  a  predicate  P  of  no  arguments,  and  let  its  intended  meaning 
be  “222  Baker  Street  Apt  13  is  unoccupied:”  that  is,  the  actual  world  satisfies  P  just  in 
case  this  apartment  is  indeed  unoccupied.  In  the  OL  there  is  also  a  predicate  symbol  P  of 
no  arguments  whose  meaning  we  wish  to  coincide  with  that  of  the  ML  predicate  P.  The 
fact  that  these  symbols  are  the  same  is  an  orthographic  accident;  they  come  from  different 
languages  and  there  is  thus  no  inherent  connection  between  them.  However,  because  the 
ML  can  describe  the  syntax  and  semantics  of  the  OL,  it  is  possible  to  axiomatize  the  desired 
connection.  Let  P'  be  the  ML  term  (in  7»  denoting  the  OL  predicate  P.  Then  P  in  the 
ML  and  OL  have  the  same  meaning  if 

PssTRUE{P')  (Rl) 

is  asserted  in  the  ML.  For  suppose  the  actual  world  satisfies  P  in  the  ML;  then  TRUE(PI ) 
must  also  hold,  and  hence  by  the  meaning  of  TRUE,  the  actual  world  is  also  a  model  for  P 
in  the  OL.  Similarly,  if  the  actual  world  falsifies  P  in  the  ML,  TRUE^no^P1))  must  hold, 
and  the  actual  world  falsifies  P  in  the  OL  also.  So  the  proposition  named  by  P'  holds  just 
in  case  Apt.  13  at  222  Baker  Street  is  unoccupied,  and  thus  the  meanings  of  P  in  the  ML 
and  P  in  the  OL  coincide. 

For  predicates  that  have  arguments,  the  connection  is  complicated  by  the  need  to  make 
sure  that  the  terms  used  in  the  ML  and  OL  actually  refer  to  the  same  individuals.  So,  for 
example,  if  P  is  a  ML  predicate  of  two  arguments  that  we  wish  to  mean  the  same  as  the 
OL  predicate  P,  we  would  write: 

TRUE{  rP(a,  0?  )sP(^(a),  A(/?));  (i?2) 

that  is,  since  the  denotation  function  A  gives  the  individuals  denoted  by  the  OL  terms  a 
and  0,  P  in  the  ML  agrees  with  P  in  the  OL  on  these  individuals.  Using  standard  names, 
R2  could  be  rewritten  as 


Vxy  P{x,  y)=TRUE(  rP(x,  y)n ) 


(*3) 

since,  by  D2,  A(rj(z))  =  x,  A(q(y))  =  y.  Note  that  the  standard  name  convention  for 
sense-quotes  is  in  force  for  R3. 

Using  TRUE  and  equivalence,  axioms  like  R3  cause  predicate  symbols  to  have  a 
“standard  meaning”  across  the  ML  and  OL,  in  much  the  same  way  that  D2  formalises 
standard  names  using  the  denotation  function  and  equality.  But  while  nonstandard  names 
are  a  useful  device  for  encoding  an  agent’s  beliefs  about  individuals  that  the  agent  may 
have  misidentified  (recall  the  Morningstar-Eveningstar  example),  nonstandard  predicates 
don’t  seem  to  serve  any  useful  purpose.  So  we  will  assume  that  for  every  predicate  symbol 
P  in  ML,  there  is  a  function  symbol  of  the  form  P1  whose  denotation  is  the  OL  predicate 
P,  and  there  is  an  axiom  of  the  form  R2  equating  the  meaning  of  these  predicates. 

To  make  the  OL  into  a  metalanguage  for  OL',  we  simply  introduce  sorts  that  denote 
OL'  expressions  into  the  OL,  tn  exactly  the  same  way  that  it  was  done  for  the  ML.  In 
addition,  the  various  axioms  that  tie  the  ML  and  OL  together  (MP,  D\,  etc.)  must  also 
be  asserted  in  the  OL.  Unfortunately,  this  also  means  that  the  ML  itself  must  have  a  new 
set  of  terms  denoting  terms  in  the  new  OL  sorts;  the  machinery  for  describing  embedded 
ML/OL  chains  rapidly  becomes  confusing  as  the  depth  of  the  embedding  grows.  So  in  this 
paper  we  will  supply  just  enough  of  the  logical  machinery  to  work  through  the  examples 
by  introducing  two  conventions;  readers  who  want  more  detail  are  referred  to  [Kon81]. 

The  first  convention  is  an  extension  of  the  sense-quote  abbreviation  to  include  ML 
variables  of  the  sort  Tp  (denoting  formulas  of  the  OL).  When  these  occur  in  sense-quotes, 
they  are  to  be  translated  as  the  standard  name  of  the  variable;  hence  they  denote  the  name 
of  an  expression.  To  take  an  example,  we  will  complete  the  axiomatization  of  CFACT : 

V/  CFACT{f)  D  Vo  KNOW(a,  rCFACT(f  )n )  (CF2) 
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CF 2  asserts  that  if  /  is  a  common  fact,  then  every  agent  knows  it  is  a  common  fact.  The 
sense-quote  term  rC F ACTt/j"1  denotes  the  OL  expression  CFACT{f),  where  J*  is  the 
standard  name  of  the  OL'  expression  corresponding  to  /. 

The  second  convention  is  to  allow  embedded  sense-quotes  to  form  the  standard  name 
of  an  expression,  as  in 

V/  CFACT{  rCFACT(f)DCFACT(  rCFACT(f f1  )n ).  (CF3) 

Here  the  embedded  sense  quotes  translate  to  the  standard  name  for  the  OL'  expression 
CFACT(f').  CF3  says  that  common  facts  will  be  inherited  down  to  the  next  level  of 
embedding  in  the  ML/OL  chain. 

In  practice,  we  hope  that  the  depth  of  embedding  needed  to  solve  a  given  problem  will 
be  small,  since  the  complexity  needed  for  even  the  three-level  structure  of  ML,  OL,  and 
OL'  is  substantial.  Also,  the  technique  of  semantic  attachment  can  be  used  to  reduce  the 
complexity  of  reasoning  about  embedded  structures  by  attaching  to  a  particular  level  of 
an  embedded  structure  and  reasoning  in  that  language.  In  Appendix  A  we  use  embedded 
ML/OL  structures  to  solve  the  wise  man  puttie,  which  involves  reasoning  to  a  depth  of 
embedding  of  three  (ML,  OL,  and  OL');  we  exploit  semantic  attachment  to  simplify  the 
reasoning  involved. 

3.3.  The  Interaction  of  Actions  and  Beliefs 

The  previous  section  laid  the  groundwork  for  a  syntactic  treatment  of  knowledge  and 
belief  in  a  static  world.  This  must  be  integrated  with  a  formal  treatment  of  actions  in  order 
to  accomplish  our  original  task  of  formalising  the  interaction  of  knowledge  and  action.  We 
examine  the  following  two  questions: 

•  What  knowledge  is  required  by  an  agent  to  successfully  perform  an  action? 

•  What  knowledge  does  an  agent  gain  in  performing  an  action? 


The  methodology  we  will  use  is  to  apply  the  situation  calculus  [McC69]  approach  to  first 
formally  describe  the  way  in  which  the  world  changes  as  events  occur.  It  will  then  be 
assumed  that  this  formal  system  is  a  reasonable  approximation  to  the  way  an  agent  reasons 
about  changes  in  the  world:  this  means  that  it  becomes  part  of  an  agent’s  rules  about 
the  world.  By  simply  attributing  a  facility  for  reasoning  about  events  to  agents,  it  turns 
out  that  we  are  able  to  answer  both  these  questions  formally,  and  that  this  formalization 
corresponds  well  with  our  intuitions  about  real  agents.  This  is  essentially  the  same  method 
that  was  used  by  Moore  in  [M008O];  here,  we  show  that  it  can  be  successfully  carried  out 
for  a  syntactic  formalization  of  knowledge  and  belief. 

Once  the  formal  requirements  for  reasoning  about  events  have  been  specified,  we 
consider  how  an  agent  might  plan  to  achieve  a  goal  using  his  knowledge  of  actions.  We 
conclude  that  planning  is  inherently  a  process  of  self-reflection:  that  is,  in  order  to  construct 
a  plan,  an  agent  must  reflect  on  what  the  state  of  his  beliefs  will  be  as  the  plan  is  undergoing 
execution.  Such  a  self-reflection  process  is  represented  naturally  by  a  ML/OL  structure  in 
which  the  planning  agent  is  identified  with  the  ML,  and  his  future  states  are  theories  of 
the  OL.  We  will  show  how  it  is  possible  to  construct  plans  within  this  representation,  and 
extend  it  to  include  plans  that  involve  other  cooperative  agents. 

3.3.1  Situations 

In  the  situation  calculus  approach,  events  are  taken  to  be  relations  on  situations,  where 
situations  are  snapshots  of  the  world  at  a  particular  moment  in  time.  It  is  natural  to 
identify  situations  with  models  of  a  language  used  to  describe  the  world;  in  this  case,  we 
will  use  the  language  OL  of  the  previous  section,  because  the  ML  for  describing  models  of 
the  OL  is  already  laid  out.  In  the  ML,  situations  will  be  named  by  terms,  generally  the 
constants  {So,5t,...}.  A  formula  /  of  the  OL  holds  in  a  situation  a  when  the  situation 
satisfies  /;  the  ML  predicate  H(s,  /)  will  be  used  to  indicate  this  condition.  If  the  situation 
Sq  is  singled  out  as  being  the  actual  world  (and  the  initial  world  for  planning  problems), 
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then  TRUE  can  be  defined  in  terms  of  H: 


VfTRUE(f)=zH(S0,f).  (HI) 

Since  H  describes  satisfiability  in  a  model,  the  truth-recursion  axioms  TR  are  valid  for  H 
as  well  as  TRUE. 

If  we  consider  agents  to  be  part  of  the  domain  of  discourse,  then  their  beliefs  can 
change  from  one  situation  to  the  next,  just  as  any  other  inessential  property  of  an  agent 
might.  But  if  an  agent’s  beliefs  change  from  situation  to  situation,  then  the  theory  that  is 
used  to  model  these  beliefs  must  also  change.  One  way  to  represent  an  agent’s  changing 
beliefs  is  to  ascribe  a  different  theory  to  an  agent  in  each  situation  to  model  his  beliefs  in 
that  situation.  In  the  ML,  we  will  write  ths(a,a)  to  denote  agent  a’s  beliefs  in  situation  a; 
if  5o  is  taken  to  be  the  actual  world,  then  it  is  obvious  that  Va  ths(a,S0)  —  th(a). 

But  we  might  now  ask  what  situation  the  expressions  in  each  of  these  theories  are 
about.  Suppose  that  the  OL  sentence  P  is  a  member  of  tha(Ao,Si),  and  thus  one  of  Ao’s 
beliefs  in  situation  5j.  We  would  naturally  want  P  to  be  a  property  that  .do  believes  to 
hold  of  situation  5t  (and  not  50  or  some  other  situation).  That  is,  ths(a,  s)  represents  agent 
a’s  beliefs  in  situation  s,  about  situation  a.  In  informal  usage  we  will  call  the  situation  we 
are  focusing  on  the  current  situation,  and  say  “the  agent  a  in  situation  a”  when  we  are 
referring  to  the  agent’s  beliefs  in  that  situation.  Later  we  will  show  how  to  represent  an 
agent’s  beliefs  about  situations  other  than  the  one  he  is  currently  in. 

For  each  situation,  an  agent’s  beliefs  in  that  situation  are  specified  by  a  theory.  Given 
this  arrangement,  we  define  the  new  predicates  B  and  K  as  similar  to  BEL  and  KNOW, 
but  with  a  situation  argument: 
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Vos/  B(a,  a,  /)  =  PR(tha(a, a),  /) 

Vas/  K(a,  s,  f)  m  B(a,  a,  f)/\H(a,  f) 

B(a,  a,  f)  means  that  in  situation  a  agent  a  believes  that  /  holds  in  a;  K  is  similar,  with  the 
condition  that  /  actually  holds  in  a.  Note  that  the  underlying  predicates  FACT  and  PR 


do  not  have  to  be  changed,  since  they  are  defined  on  theories  of  OL  rather  than  models. 
Thus  the  properties  of  BEL  and  KNOW  described  in  the  previous  section  also  hold  for  B 
and  K  in  any  particular  situation.  BEL  and  KNOW  can  be  defined  as  B  and  K  in  the 
situation  So- 

Several  extensions  to  the  formalism  presented  in  the  first  section  must  be  made  to  deal 
with  situations.  A  new  denotation  function  6  takes  a  situation  argument  as  well  as  an  OL 
term:  6(9,  a)  is  the  denotation  of  a  in  situation  a.  A(a)  gives  the  denotation  of  a  a  in 
situation  So,  and  is  definable  as  £(S0, <*)•  The  appropriate  forms  of  D 1  and  D2  are: 


H(  ra  -  p )  a  [S (a,  a)  =  6(s,  0)} 
Vax  6(a,ti(x))  =  x 


(D3) 


This  last  says  that  standard  names  always  have  the  same  interpretation  in  every  situation. 
Non-standard  names  can  change  their  denotation  in  different  situations,  e.g.,  the  block 
denoted  by  “the  block  Ao  is  holding”  may  be  changed  by  Ao’s  actions. 

Finally,  we  require  the  appropriate  versions  of  R1-R6,  where  these  axioms  are  ap¬ 
propriately  generalized  to  refer  to  all  situations. 


3.3.2  Observables 

Following  Moore  [M008O],  we  recognize  three  ways  that  an  agent  can  acquire  beliefs  in  a 
situation: 

•  He  can  observe  the  world  around  him. 

•  His  beliefs  about  past  situations  persist  in  the  current  situation. 

•  He  can  reason  about  the  way  in  which  the  current  situation  arose  from  events  that 
occurred  in  previous  situations. 


In  the  next  few  subsections  we  describe  how  an  agent’s  beliefs  persist  and  how  he  reasons 
about  events;  here  we  formalize  what  it  means  for  a  property  of  the  world  to  be  observable. 
It  is  certainly  true  that  there  are  many  properties  of  the  world  we  live  in  that  are  not 


directly  observable;  for  example,  consider  a  gas  oven  whose  pilot  light  is  completely  encased 
and  hence  not  visible.  Whether  this  pilot  light  is  on  or  off  isn’t  an  observable  property,  but 
there  are  other  observations  that  could  be  made  to  test  what  the  state  of  the  pilot  light  is, 
e.g.,  by  turning  on  the  oven  and  observing  whether  it  lights.  What  we  actually  consider  to 
be  observable  depends  on  how  we  formalize  a  given  problem  domain;  but  it  is  important 
for  a  planning  agent  to  be  able  to  make  the  distinction  between  properties  of  the  world  he 
can  observe  directly,  and  those  he  must  infer. 

One  of  the  reasons  that  it  is  handy  to  have  a  separate  theory  representing  the  beliefs  of 
an  agent  in  each  situation  is  that  we  then  have  a  way  of  describing  the  effect  of  observable 
properties  on  an  agent’s  beliefs.  Formally,  we  can  state  that  a  property  is  observable  by 
asserting  that  in  every  situation,  subject  to  certain  preconditions  that  are  required  for  the 
felicitous  observation  of  the  property,  an  agent  knows  whether  that  property  holds  or  not. 
For  example,  in  the  OL  let  o  be  an  oven,  and  let  LIT{o)  mean  that  o  is  lit.  Then  LIT(o)  is 
asserted  to  be  observable  by: 

Vaos  H(s,  rAT(a,  of )  D  [ K(a,  *,  rL/r(of  )VK(a,  a,  r~L/r(of ) ]  (Ol) 

that  is,  if  the  agent  is  actually  at  the  oven,  he  knows  either  that  it  is  lit,  or  that  it  is 
not  lit.  Recall  from  the  previous  section  on  knowledge  and  belief  that  this  says  something 
very  strong  about  the  state  of  a’s  knowledge,  and  is  not  derivable  from  the  tautology 
K(a,a,  rLIT( o )  V~ LIT( o f ). 

3.3.3  Events  Types 

Event  types  are  relations  on  situations;  a  given  event  type  describes  the  possible  states 
of  the  world  that  could  result  from  an  event  occurring  in  any  initial  state.  We  will  use  the 
three-place  predicate  EV  in  the  metalanguage  to  describe  event  types:  EV(e, a/),  where 
e  is  an  event  type  and  «,•  and  «/  are  situations,  means  that  «/  results  from  an  event  of  type 


e  occurring  in  An  event  is  an  instance  of  an  event  type,8  but  generally  we  will  not  have 
to  distinguish  them  for  the  purposes  of  this  paper,  and  we  wilt  use  “event”  for  “event  type” 
freely. 

Generally  the  events  of  interest  will  be  agents’  actions,  and  these  will  be  constructed 
in  the  ML  using  terms  representing  actions,  agents,  and  the  objects  involved  in  the  action 
(the  parameters  of  the  action).  If  act  is  an  action,  then  do(a,act)  is  the  event  of  agent  a 
performing  this  action.  Consider  the  situation  calculus  axiomatization  of  a  simple  blocks- 
world  action,  puton(x,y),  where  the  parameters  of  the  action  are  blocks: 


VaxyaiSf  EV(do(a,puton(x,it)),8i,8f)  D  rCLEAR{yf')A 

H(sit  rHOLDING(a,  xf  )A 
H{a/,r0N(9ty?) A 
H{sf,  r~HOLDING(a,xf)  A 


C POl ) 


Vflxys,«/  EV(do(a,puton(x,y)),«i, «/)  D 

[  V/  SAF(f)Af  ^  rCLEAR(yf  A /  ^  rHOLDING(a,  xf  D  (F02) 
H(s<,f)=H(sf,f)\ 

The  form  of  POl  is  an  implication,  so  the  right-hand  side  describes  the  conditions  under 
which  situations  and  «/  are  related  by  the  event  of  a  putting  x  on  y.  The  first  two 
conjuncts  on  the  right-hand  side  are  essentially  preconditions  for  the  event  to  occur,  since 
they  state  conditions  on  the  initial  situation  that  must  be  satisfied  for  EV  to  hold.  The 
preconditions  are  that  C LEAR(if(y))  and  HOLDING(i\(a),i\(x ))  must  hold  in  situation 
note  that  the  standard  names  for  the  parameters  are  indicated  by  the  sense-quote 
convention.  If  the  preconditions  are  not  met,  then  there  is  no  situation  «/  that  is  the 
successor  to  under  the  event  e.  The  rest  of  the  conjuncts  describe  which  formulas  of  the 
OL  are  to  hold  in  the  new  situation  «/. 


9 For  example,  “Borg’s  winning  of  Wimbledon  yesterday  was  fortuitous"  is  a  statement  about  a  single  event,  but 
“Borg  winning  Wimbledon  has  happened  five  times"  describes  an  event  type  that  had  five  particular  instances. 


P02  specifies  that  all  formulas  of  a  certain  type  that  hold  in  are  also  to  hold  in 
8/.  It  is  thus  a  frame  axiom  for  the  event  e,  describing  which  aspects  of  the  situation 
remain  unchanged  after  the  event  occurs.  The  predicate  SAF  stands  for  Simple  Atomic 
Formula;  it  picks  out  those  formulas  of  the  OL  that  are  composed  of  atomic  predicates 
over  standard  names.  Although  SAF  applies  only  to  nonnegated  atomic  formulas,  the 
frame  axiom  carries  over  negated  atomic  formulas  as  well,  since  //(«,  not(/))  is  equivalent  to 
f).10  Among  the  nicer  features  of  this  axiomatization  is  that  events  whose  outcomes 
are  conditional  on  the  initial  state  can  be  easily  described.  For  instance,  consider  the  event 
of  an  agent  turning  on  a  gas  oven  that  has  a  pilot  light.  If  the  pilot  light  is  on,  the  oven  will 
be  lit;  if  the  pilot  light  is  off,  the  oven  will  have  whatever  status,  lit  or  unlit,  it  had  before 
the  event  occurred  (the  oven  may  already  have  been  on).  Let  PL(o)  be  an  OL  predicate 
meaning  “the  pilot  light  of  oven  o  is  on”;  and  let  LIT(o)  mean  “oven  o  is  lit.”  Then  the 
event  of  an  agent  turning  on  o  can  be  described  as: 


Vas,«/o  EV(do(o,light(o)),  s,-,s/)  3 

H(si,rAT(a,o)')  A 

H(»it  rPU(oV rLIT(oV )  A 

#(*,-,  r~PHoD  D  rLIT(op )=H(ai,  rLIT(o j1)] 


Va«,s/o  EV(do(a,light(o)),  s,-, «/)  D 

[V/  SAF(f) A/  ¥>  rLIT{oV  D  H(*i,f)mH(sf,f)) 


(LT2) 


The  second  conjunct  of  LTl  gives  the  result  of  the  event  in  case  the  pilot  light  is  on:  the 
oven  will  be  lit.  The  third  conjunct  says  that  if  the  pilot  light  is  off,  the  oven  will  be  lit  in 


8/  just  in  case  it  was  lit  in  i.e.,  its  status  doesn’t  change.  LT2  is  the  frame  axiom. 


3.3.4  Reasoning  about  Situations  and  Events 

The  axiomatization  of  events  as  relations  on  situations  enables  us  to  talk  about  what  is 


true  in  the  world  after  some  events  have  occurred  starting  from  an  initial  situation  (which 
we  will  generally  take  to  be  So).  What  it  doesn’t  tell  us  is  how  an  agent’s  beliefs  about  the 
world  will  change;  nothing  in  the  PO  or  LT  axioms  gives  any  insight  into  this.  It  might 
be  suspected  that,  as  events  are  described  by  axioms  as  changing  the  actual  state  of  the 
world,  this  description  might  be  extended  to  cover  agents’  theories  as  well,  e.g.,  changing 
^4o’s  theory  in  situation  So  ( ths{Ao , So))  into  his  theory  in  situation  Si  (tha(Ao, Si)).11  But 
there  is  no  obvious  or  well-motivated  way  to  make  modifications  to  axioms  like  PO  and  LT 
so  that  they  take  into  account  agents’  beliefs  about  a  situation  rather  than  what  actually 
holds  in  the  situation.12  What  is  needed  here  is  a  principled  way  of  deriving  the  changes 
to  an  agent’s  beliefs  that  result  from  an  event,  given  a  description  of  the  event  as  a  relation 
on  situations.  Credit  for  the  recognition  of  this  problem  belongs  to  Robert  Moore,  and  we 
will  formalize  the  solution  he  presented  in  his  thesis,  the  main  points  of  which  follow. 

The  solution  to  this  difficulty  lies  in  making  the  observation  that  agents  are  reasoning 
entities.  Consider  how  agent  Aq  might  reason  about  some  event  E;  let  us  suppose  the 
event  is  that  agent  Aq  turned  on  the  oven  in  situation  So,  and  that  the  result  was  that 
the  oven  was  not  lit  in  situation  Si.  What  should  j4o’s  beliefs  be  in  situation  Si?  First, 
by  observation,  he  knows  that  the  oven  isn’t  lit.  He  also  believes  (in  Si)  that  the  current 
situation  resulted  from  the  event  E  occurring  in  situation  S0.  So  Aq  reasons  as  follows:  if, 
in  situation  S0,  the  pilot  light  of  the  oven  had  been  on,  then  in  St  the  oven  would  be  lit, 
since  he  turned  it  on.  But  the  oven  isn’t  lit;  hence  the  pilot  light  couldn’t  have  been  on  in 
So,  and  remains  not  on  in  S\. 

There  are  several  important  things  to  note  about  this  analysis.  The  first  is  that,  as 

11  Indeed,  it  might  be  thought  that  the  most  widely  known  AI  planning  system,  STRIPS,  has  just  such  a  mechanism 
in  the  its  add/delete  list  approach  to  describing  events.  However,  closer  examination  reveals  that  because  ST  RI  PS 
makes  the  assumption  that  it  has  a  partial  model  in  the  sense  of  [Wey80],  it  is  actually  slightly  less  descriptive  than 
the  situational  approach  described  above  (NilSO]. 


13  There  is  one  proposal  that  is  suggested  by  the  our  use  of  H  to  refer  to  the  actual  situation  and  PR  to  statements 
that  an  agent  believes  about  a  situation,  namely,  to  replace  ail  predicates  involving  H  with  the  corresponding 
ones  involving  PR.  However,  it  can  be  shown  that  the  substitution  of  PR(tht(A0,t),...)  for  yields 

counterintuitive  results  tor  A0 's  beliefs. 
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suggested  previously,  Ao's  beliefs  in  situation  Si  comes  from  only  three  sources:  observa¬ 
tions  ( “the  oven  is  not  lit” ),  persistence  of  beliefs  about  previous  situations  ( “if  in  So  the 
pilot  light  had  been  on. . .” ),  and  beliefs  about  the  way  events  change  the  world.  This  latter 
is  equivalent  to  having  some  form  of  PO 1  as  part  of  Ao’s  beliefs  in  situation  S\.  From 
these  three  sources  Ao  is  able  to  generate  a  new  set  of  beliefs  for  Sj. 

The  second  thing  to  note  is  that  none  of  Ao’s  reasoning  in  Si  could  have  taken  place 
unless  he  believed  that  Si  resulted  from  So  via  the  event  E.  Beliefs  about  what  sequence 
of  events  led  to  the  current  situation  play  a  very  important  role  in  reasoning  about  that 
situation,  and,  like  other  beliefs,  they  can  be  mistaken  or  inferred  from  other  evidence. 
Suppose,  for  example,  that  Ao  suddenly  sees  the  oven  become  lit.  He  might  infer  that  the 
only  way  that  could  happen  when  it  wasn’t  previously  lit  would  be  for  an  agent  to  turn  it 
on;  this  is  inferring  that  the  situation  where  the  oven  is  lit  is  connected  by  a  certain  event 
with  a  previous  situation  where  the  oven  wasn’t  lit.  We  will  not  be  concerned  with  this 
kind  of  inference  here,  although  we  note  the  possibility  of  doing  event  recognition  in  this 
framework.  The  events  we  are  interested  in  are  actions,  and  the  assumption  we  will  make 
for  the  remainder  of  this  paper  is  that  an  agent  knows  what  action  it  is  that  he  performs 
in  executing  a  plan. 

A  third  aspect  of  this  reasoning  that  is  unusual  is  that  the  axiomatization  of  events 
is  being  used  in  a  different  way  than  a  planning  program  would  normally  consider  doing. 
Typically,  a  planner  uses  an  event  description  like  LT1  to  form  plans  to  light  the  oven,  and 
the  side  condition  that  the  pilot  light  be  on  is  one  of  the  things  that  can  go  wrong  with 
the  plan,  and  so  must  be  taken  into  account  as  a  subgoal.  However,  in  the  above  example 
Ao  has  used  LT 1  to  reason  about  a  property  of  the  world  that  is  not  available  to  his  direct 
observation,  that  is,  as  a  test.  This  is  an  important  characteristic  for  any  formalism  that 
combines  a  description  of  agents'  beliefs  with  a  description  of  events;  a  single  description 
of  an  event  should  suffice  for  an  agent  to  reason  about  it  either  as  a  means  of  effecting  a 
change  in  the  world,  or  as  a  test  that  adds  to  his  beliefs  about  the  world. 


Finally,  the  precondition  that  Aq  be  at  the  oven  to  turn  it  on  translates  naturally  in 
this  analysis  into  a  precondition  on  Ao’s  beliefs  in  situation  So.  If  Ao  is  to  reason  that 
situation  Si  is  the  successor  to  So  under  the  event  E,  he  must  believe  that  he  was  actually 
at  the  oven  in  situation  Sq.  For  if  he  doesn’t  believe  this,  then  he  cannot  use  PO 1  to  infer 
anything  about  the  results  of  his  action. 

We  might  summarize  the  analysis  of  this  section  in  the  following  way:  by  making  the 
simple  assumption  that  an  agent  reasons  about  the  way  in  which  situations  are  related  by 
events,  we  are  able  to  characterize  in  a  natural  way  the  belief  preconditions  required  for 
executing  an  action,  and  the  effects  of  actions  on  the  subsequent  belief  state  of  an  agent. 

The  interaction  of  observation  and  reasoning  about  situations  gives  an  agent  the  power  to 
plan  actions  that  perform  tests,  as  well  as  change  the  state  of  the  world. 

3.3.5  Formalizing  Agents’  Reasoning  about  Events 

We  now  give  a  formalization  that  implements  the  ideas  just  laid  out.  The  first 
requirement  is  that  we  be  able  to  describe  an  agent  a  in  situation  s  reasoning  about  other 
situations,  especially  the  one  just  preceding.  Since  the  formulas  of  ths{a,a)  all  refer  to 
properties  of  situation  s,  we  must  enrich  the  OL  so  that  formulas  in  the  OL  can  refer  to 
different  situations.  Using  the  techniques  of  belief-nesting  of  the  previous  section,  we  add 
to  the  OL  the  predicate  H  corresponding  to  the  ML  predicate  of  the  same  name.  Then  the 
OL  expression  H(S\ ,  rP1 )  means  that  the  OL'  formula  P  holds  in  situation  S\ ,  regardless 
of  what  theory  this  formula  appears  in.13  With  the  addition  of  the  H  predicate  to  the  OL, 
the  notion  that  all  formulas  in  the(a,  a)  refer  to  properties  of  a  can  be  formalized  as: 

Vs/  PR(th(a,  a),  /)  =  PR(th(a,  a),  rH(a,  /)n )  (i/2) 

H 2  can  be  paraphrased  by  saying  that  an  agent  believes  P  in  situation  a  just  in  case  he 

13  We  will  take  {50,51 to  be  standard  names  for  situations  in  all  languages.  It  will  be  assumed  that  standard 
names  are  always  used  to  name  situations. 


believes  that  P  holds  in  situation  a.  Given  H 2,  it  is  possible  to  describe  agents’  theories  as 
consisting  purely  of  formulas  in  H;  but  the  added  level  of  embedding  puts  this  technique 
at  a  disadvantage  with  respect  to  using  other  predicates  from  OL  to  describe  an  agent’s 
beliefs  about  the  current  situation. 

It  is  also  possible  to  formalize  the  notion  that  beliefs  about  previous  situations  persist, 
or  are  carried  over  into  succeeding  situations.  Suppose  that  in  situation  Sn  an  agent  has  a 
belief  of  the  form,  “in  a  previous  situation  Si,  P  was  true.”  Then  if  £n+i  is  the  successor 
to  Sn  under  some  event,  this  belief  is  still  valid.  Formally,  we  can  assert  this  with  the  ML 
axiom: 

Vs.s/c  EV(c,  Si,  sf)  D  ( VasfB(a ,  sit  rH(s,  /)n  )DB(a,  sf,  rH{s,  /j1 )]  (HZ) 

The  antecedent  of  the  implication  says  that  a,-  and  «/  must  be  connected  by  some  event 
for  beliefs  to  be  carried  over  from  «,  to  a/;  this  is  necessary  because  we  don’t  want  agents 
to  inherit  beliefs  from  their  future  states.  By  phrasing  the  beliefs  in  terms  of  the  predicate 
H,  H3  carries  over  beliefs  about  all  situations  previous  to  and  including 

One  of  the  consequences  of  HZ  is  that  once  an  agent  forms  a  belief  about  a  situation, 
he  holds  that  belief  about  that  situation  for  all  time.  Since  beliefs  can  be  mistaken,  it 
might  happen  that  an  agent  observes  something  that  forces  him  to  revise  his  previously 
held  beliefs.  In  that  case,  HZ  is  too  strong,  and  the  resultant  theory  will  be  inconsistent. 

We  recognize  that  the  general  problem  of  reconciling  inconsistent  beliefs  that  arise  from 
different  sources  (called  belief  revision)  is  a  hard  one,  involving  both  conceptual  and  tech¬ 
nical  issues,  and  it  is  not  part  of  this  research  to  say  anything  new  about  it.14  Nevertheless, 
it  is  worthwhile  to  note  that  because  the  ML  has  terms  that  refer  to  agents’  theories  in 
different  situations,  it  may  be  possible  to  describe  a  belief  revision  process  formally  in  the 
ML. 

14  Doyle  [Doj78]  worked  on  this  problem  under  the  rubric  “Truth  Maintenance,*  and  more  recent  work  in  non¬ 
monotonic  reasoning  also  considers  this  problem. 
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3.3.6  An  Example  of  a  Test 

Given  the  preceding  techniques  for  describing  what  an  agent  believes  to  hold  in  situar 
tions  other  than  the  one  he  is  currently  in,  we  can  show  formally  that  Aq  can  use  the  LT 
axioms  as  a  test  to  figure  out  whether  the  pilot  light  is  on  or  not.  In  the  initial  situation 
So,  we  will  assume  that  Ao  knows  he  is  at  the  oven  O  (where  O  is  the  standard  name  for 
the  oven),  and  realizes  that  it  is  not  lit: 

Initial  Conditions,  in  the  ML_ 

(1)  K(Ao,S0,rAT(Ao,O)A~~LIT(O)])  given 

(2)  K(Ao,Si,rEV(do(Ao,light(O)),S0,Sip)  given 

The  style  of  proof  we  will  exhibit  will  be  natural  deduction,  with  assumption  dependencies 
noted  in  square  brackets  in  the  justification  for  a  line  of  the  proof.  Given  the  initial 
conditions,  we  next  show  that  Ao  can  observe  whether  or  not  the  oven  is  lit  in  situation 

Sf. 


(3)  V/  SAF(f)Af  jk  rLIT(OV  D  H(S0,  /)si/(S, ,  /) 

(4)  SAF(rAT(Ao,OD 

(5)  H(Sx,rAT(Ao,OV) 

(0)  K(Ao,  Si ,  rLIT(0 )n  )vK(Aa ,  5, ,  r~L/T(0)n ) 


2,B2,LT2 
definition  of  SAF 
i,3,4,B2 
5,01 


Line  3  comes  from  the  frame  axiom  for  light,  and  lets  us  infer  that  Ao  is  still  at  the  oven 
in  situation  S\  (line  5).  The  observation  axiom  01  is  then  invoked  to  assert  that  Ao  will 
know  what  the  state  of  the  oven  is  in  that  situation. 


Throughout  this  proof,  we  will  be  interested  in  two  theories  of  the  OL:  ths(Ao,  So)  and 
ths(Ao,Si).  Assertions  in  the  ML  involving  Ao’s  beliefs  can  be  reasoned  about  by  using 


semantic  attachment  to  the  appropriate  OL  theory.  For  example,  line  1  above  is  attached 
to  the  following  statements  in  th«(Ao,So): 


Ap's  Theory  in  Situation  Sp 

(7)  AT(Ao,0)A~LIT(C>)  1,B2, semantic  attachment 

(8)  H(S0,r~LIT(O)])  l,B2,H 2, semantic  attachment 

Line  7  is  the  attachment  of  line  1  to  Ao’s  theory  in  So.  Line  8  is  derived  from  line  1  by 
the  use  of  H 2;  it  is  useful  because  it  will  persist  as  a  belief  in  the  successor  situation  St. 
Generally,  beliefs  that  an  agent  derives  about  the  current  situation  can  be  inherited  into 
succeeding  situations  by  expressing  these  beliefs  with  the  //  predicate. 

At  this  point  we  do  reasoning  by  cases.  First  assume  the  right  disjunct  of  line  0;  then 
for  Ao’s  beliefs  in  situation  Si  we  have: 

Ao’s  Theory  in  Situation  S| 

(9)  ~LIT{0)  [9] :assumed, semantic  attachment 

(10)  ~-f/(Si,  rZ,/T(0)n)  [9]:9,//2semantic  attachment 

(11)  EV{do{Ao,light(0)),  S0,  Sx)  2, semantic  attachment 

(12)  //(S0,  rPL(OT)DH(Sl,  rLIT(0?)  11, m 

(13)  --•//(So,  rPl(Op)  [9):10, 12  contrapositive 

(14)  //(S0,r~PL(Or)  [9]:13,77J  for  H 

The  first  part  of  the  result  is  derived  by  line  14,  namely,  that  if  Ao  observes  that  the  O  is 
not  lit  in  situation  So,  then  he  knows  that  the  pilot  light  was  not  on  in  situation  So.  This 
sequence  of  steps  is  interesting  because  it  illustrates  the  intermixture  of  proof  techniques 
in  the  ML  and  OL.  Lines  9,  10,  and  11  come  from  statements  in  the  ML  about  tA«(Ao,  Si). 
Line  10  is  derived  from  line  9  in  the  ML  by  the  application  of  axiom  H2.  Line  11  says  that 


v4o  believes  that  5j  is  the  result  of  the  light(0)  action  occurring  in  5o,  and  follows  directly 
from  line  2  and  semantic  attachment.  Line  12  follows  from  line  11  and  the  event  axiom 
LTV,  it  is  assumed  that  Ao  believes  this  axiom.  Finally,  13  and  14  follow,  given  that  the 
truth-recursion  axioms  for  H  are  made  available  in  all  theories  in  the  OL. 

The  left  disjunct  of  line  6  can  be  reasoned  about  in  the  following  way  (since  lines  11 
and  12  did  not  involve  any  assumptions,  they  can  be  used  in  this  part  of  the  proof  also): 


Aq's  Theory  in  Situation  Si 

(15)  LIT(0)  [I5]:assumed,  sem.  att. 

(16)  H(SX,  rLIT{OV)  (l5]:15,H2,sem.  att. 

(17)  ~tf(S0,  rLIT(0?)  S,HZ,TR  for  H, sem.  att. 

(18)  ~[tf(Si,  rL/r(O)n)=tf(S0,  rLITiO)')\  [151:10,17 

(19)  H(S0,  r~PU(OT)  Z>  H(SX,  rLIT{OF)mH{So,  rLIT(Of)  U,LTl 

(20)  ~f/(So,  r'—PL(Oj1)  [15]:18, 19  contrapositive 

(21)  H{S0,  rPL(Of )  [15]:20,Tf?  for  H 


Here  again,  the  first  few  lines  (15,  16  and  17)  are  established  by  reasoning  at  the  ML  about 
th8{Ao,Si).  Line  17  comes  from  an  instance  of  axiom  HZ,  which  enables  an  agent’s  beliefs 
to  persist  through  a  sequence  of  situations.  Line  19  comes  from  .Ao’s  knowledge  of  LT 1, 
and  line  20  is  the  key  step:  it  establishes  that  under  the  assumption  of  O  being  lit  in  Sx, 
the  pilot  light  was  on  in  So.  Finally,  the  frame  axiom  LT2  will  carry  the  pilot  light’s  status 
in  So  forward  into  Si: 
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Ao’s  Theory  of  Situation  Si 


(22)  WfSAF(f)A/^rLIT(0)]DH(So,f)=H(S1,f)  11  ,LTl 

(23)  SAF(  rPL(OV )  definition  of  SAF 

(24)  H(S0,rPL(O)1)sff(S1,rPL(O)1)  22,23 

(25)  Pl\0)  [l5]:21,24,i/2 

(26)  ~PL(0)  [9]:14,24,H2 

Line  25  is  under  the  assumption  of  the  left  disjunct  of  line  6,  and  line  26  is  under  the  right 
disjunct.  In  the  ML  we  can  derive  several  results  from  the  preceding  proof  structure: 


In  the  ML 

(27)  B(Ao,Su  rPL(0)')VB(AatSi,  r~PHO?)  6,20,21 

(28)  B(Aq, Si ,  rLIT(Of )  D  B(Ao, S, ,  rPL(0)' )  15,25 

(29)  B(A0,St,r~LIT{0)')DB{A0,Sl,r~PlJ(O)])  9,26 

Line  27  says  that  in  Si,  Ao  will  either  believe  that  the  pilot  light  is  on,  or  he  will  believe 
that  is  not  on.  Thus,  by  performing  the  action  of  lighting  the  oven,  Ao  gains  knowledge 
about  the  state  of  an  unobservable,  the  pilot  light.  This  is  the  desired  result  of  agent  Ao 
using  LTl  to  perform  a  test  of  an  unobservable  property. 

Lines  28  and  29  give  proof-theoretic  analogues  to  the  LT  axioms,  which  described 
the  event  of  lighting  the  oven  solely  in  terms  of  the  actual  situations  before  and  after  the 
event.  These  assertions  show  how  the  beliefs  of  Ao  change  under  the  influence  of  the  event 
do(a,light(0)).  By  suitably  generalizing  the  preceding  proof,  it  can  be  shown  that  28  and 
29  hold  for  all  agents  and  initial  situations: 

VaoaiSf  EV(do(a,  light(o)), «/)A rAT{a,  o)A~L/T’(o)_i  )  D 

B(a,  »f,  rLIT(o j1 )  D  B(a,  */,  rPL(o)1 ) 

B(a,Bf,  r~LIT{o)])D  B(a ,  Bf,  ^PHo?) 

{LTZ) 
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LTZ  is  valid  under  the  condition  that  LT 1  is  assumed  to  be  believed  by  all  agents.  LTZ  is 
one  description  of  the  way  in  which  an  agent’s  beliefs  change  in  a  situation  that  results  from 
an  oven-lighting  event;  it  would  be  most  useful  to  a  planner  as  a  lemma  to  be  invoked  if  the 
state  of  the  pilot  light  were  to  be  tested  as  a  step  in  a  plan.  Another  lemma  about  oven¬ 
lighting  that  would  be  useful  to  a  planner  would  be  one  in  which  the  belief  preconditions 
to  an  action  were  made  explicit;  this  would  be  used  to  plan  actions  that  light  the  oven. 

3.3.7  Plans  and  Planning 

In  the  previous  subsection  we  saw  how  to  characterize  the  changes  to  an  agent’s  beliefs 
produced  by  his  observations  of  events.  In  this  subsection  we  will  consider  how  to  use  these 
results  as  part  of  the  deductions  that  an  agent  needs  to  do  to  construct  workable  plans, 
i.e.,  plans  that  will  accomplish  their  goals. 

Consider  how  an  agent  might  go  about  constructing  workable  plans.  Using  his  descrip¬ 
tion  of  various  events  (PO,  LT,  and  others)  he  can  try  to  find  a  sequence  of  actions  that 
lead  to  the  desired  goals  being  true  in  some  final  situation.  If  we  identify  the  planning 
agent  with  the  ML,  then  a  plan  would  be  a  sequence  of  situations  connected  by  actions 
performed  by  that  agent,  such  that  the  goals  are  true  in  the  final  situation.  This  doesn’t 
seem  to  involve  the  planning  agent  in  any  reasoning  about  his  beliefs;  all  he  needs  to  do  is 
describe  how  the  actual  world  changes  under  the  influence  of  his  actions. 

This  isn’t  the  whole  story,  though.  The  plan  that  is  derived  must  be  an  executable 
plan;  that  is,  if  the  plan  is  a  sequence  of  actions,  the  agent  must  be  able  to  execute  each  of 
those  actions  at  the  requisite  time.  For  instance,  the  action  description  light(oven(John)) 
will  not  be  executable  if  Aq  doesn’t  know  which  oven  is  John’s.  For  a  plan  to  be  executable 
by  an  agent,  the  agent  must  know  what  action  is  referred  to  by  each  of  the  do-terms  in  the 
plan.  According  to  a  previous  section,  this  means  that  the  agent  must  have  the  standard 
name  for  the  action  in  his  theory.  But  what  are  standard  names  for  actions?  Following 
Moore  [M008O],  we  take  the  viewpoint  that  actions  can  be  analyzed  as  a  general  procedure 


applied  to  particular  arguments,  e.g.,  puton  is  a  general  procedure  for  putting  one  block 
on  top  of  another,  and  puton(A,  B)  is  that  procedure  applied  to  the  two  blocks  A  and  B. 
If  we  assume  that  all  agents  know  what  general  procedure  each  action  denotes,  then  the 
standard  names  for  actions  are  simply  the  terms  formed  by  the  action  function  applied  to 
the  standard  names  of  its  parameters.15  The  condition  that  actions  be  executable  forces 
the  planning  agent  to  make  the  critical  distinction  between  his  beliefs  at  planning  time  and 
his  beliefs  at  execution  time.  A  planning  agent  may  not  know,  as  he  forms  his  plan,  exactly 
what  action  a  particular  do- term  in  his  plan  denotes;  but  if  he  can  show  that  at  the  time 
he  is  to  execute  that  action,  he  will  know  what  it  is,  then  the  plan  is  an  executable  one. 
Plans  of  this  type  occur  frequently  in  common-sense  reasoning;  consider  a  typical  plan  Ao 
might  form  to  tell  someone  what  time  it  is.  The  plan  has  two  steps:  first  Ao  will  look  at 
his  watch  to  find  out  what  the  time  is,  and  then  he  will  communicate  this  information  to 
the  requester.  At  planning  time,  Ao  doesn’t  really  know  what  the  second  action  is,  because 
he  doesn’t  know  the  time,  and  the  time  is  an  important  parameter  of  the  communication 
act.  Yet  he  can  reason  that  after  looking  at  Us  watch,  he  will  know  the  time;  and  so  the 
plan  is  a  valid  one. 

By  this  argument,  an  agent  must  analyze  at  planning  time  what  the  future  states  of 
his  beliefs  will  be  as  he  executes  the  plan.  Thus  the  planning  process  intrinsically  forces 
the  agent  into  introspection  about  his  future  beliefs.  Since  we  have  identified  the  planning 
agent  with  the  ML,  it  is  natural  to  represent  his  future  beliefs  during  the  execution  of 
the  plan  as  OL  theories  in  the  situations  that  the  planning  process  gives  rise  to.  If  the 
planning  agent  is  Ao,  then  these  theories  are  fA«(Ao,  Sq)  (the  initial  situation),  ths(Ao,  Si), 
etc.,  where  each  of  the  S,-  results  from  its  predecessor  via  the  execution  of  the  next  action 


18  Actually,  the  condition  that  the  parameters  be  standard  names  is  too  strong.  Standard  names  have  the  property 
that  every  agent  knows  whether  two  individuals  named  by  standard  names  are  the  same  or  not  in  every  situation,  but 
this  condition  is  not  strictly  necessary  for  an  action  to  be  executable.  Consider  the  action  of  requesting  information 
from  the  telephone  operator;  surely  it  is  not  required  that  an  agent  be  able  to  differentiate  the  operator  from  every 
other  individual  in  his  beliefs.  If  he  were  to  dial  the  operator  on  two  separate  occasions,  he  would  not  necessarily  be 
able  to  tell  if  he  talked  to  the  same  operator  or  not. 
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in  the  plan.  Ao’s  planning  process  is  basically  a  simulation  of  the  plan’s  execution  in  which 
he  reasons  about  the  changes  that  both  the  actual  world  and  his  set  of  beliefs  will  undergo 
during  the  course  of  the  plan’s  execution.  By  figuring  out  what  his  future  states  of  belief 
will  be,  he  can  decide  at  planning  time  whether  an  action  of  the  plan  will  be  executable. 

For  Aq  to  take  other  agents’  plans  into  account  in  forming  his  own,  he  must  be  able  to 
represent  their  future  states  of  belief,  in  addition  to  his  own.  But  this  doesn’t  involve  any 
additional  representational  complexity,  since  Aq  is  already  keeping  track  of  his  own  beliefs 
during  the  simulated  execution  of  the  plan.  In  [Kon80j  an  example  of  a  multiagent  plan  is 
presented;  currently  we  are  working  on  formalizing  such  plans  in  the  framework  presented 
here. 

Actually,  this  planning  process  bears  a  strong  resemblance  to  typical  implementa¬ 
tions  of  a  situation  calculus  approach  to  planning  [War74].  In  these  systems,  events  are 
axiomatized  along  the  lines  of  PO  and  LT,  and  the  planner  searches  for  a  sequence  of 
situations  that  leads  to  the  goal  by  doing  theorem-proving  with  the  event  axioms;  the 
search  space  is  essentially  the  same  in  either  approach.  The  main  difference  is  in  the  rela¬ 
tive  complexity  of  reasoning  that  the  two  planning  systems  must  be  able  to  handle.  In  the 
approach  described  here,  the  effect  of  actions  on  the  agent’s  beliefs  in  each  situation  greatly 
increases  the  deductive  complexity  of  the  planner  and  the  work  that  it  must  do  at  each 
node  in  the  search  space  of  plans.  The  usefulness  of  lemmas  such  as  LT3  that  describe 
the  effects  of  actions  on  an  agent’s  belief  state  now  becomes  apparent:  by  summarizing  the 
effect  of  actions  on  an  agent’s  beliefs,  they  reduce  the  complexity  of  the  deductions  that 
must  be  performed  at  each  step  in  the  plan.  Further  savings  can  be  realized  by  using  the 
method  of  belief  attachment  described  in  the  previous  section:  from  at  the  ML, 

infer  K{Aq,  «,  /).  Most  of  the  work  of  figuring  out  Ao’s  future  states  of  knowledge  can  be 
performed  by  reasoning  about  H  at  the  metalevel,  rather  than  K,  and  this  is  considerably 
simpler.  Finally,  it  should  be  noted  that  the  executability  requirement  acts  as  a  filter  on 
plans.  Thus  a  reasonable  search  strategy  would  be  to  first  find  a  plan  that  works  without 


taking  into  account  its  executability  (and  hence  the  future  belief  states  of  the  planning 
agent),  and  then  test  it  for  executability. 

3.3.8  Conclusion 

To  summarize  the  contributions  of  this  paper:  we  have  defined  a  syntactic  approach 
to  the  representation  of  knowledge  and  belief  in  which  the  key  element  is  the  identification 
of  beliefs  with  provable  expressions  in  a  theory  of  the  object  language.  The  technique 
of  semantic  attachment  to  the  intended  interpretation  of  the  metalanguage  provability 
predicate  has  been  advanced  as  a  method  of  simplifying  proofs  by  directly  modeling  an 
agent’s  inference  procedure,  rather  than  simulating  it. 

To  unify  a  formalization  of  knowledge  and  action,  we  have  shown  how  to  take  Moore’s 
account  of  their  interaction  and  formalize  it  within  the  syntactic  framework.  The  benchmark 
example  was  a  presentation  of  a  test  in  which  an  agent  uses  his  knowledge  of  observable 
properties  of  the  world  and  the  way  actions  affect  the  world  to  discover  the  state  of  an 
unobservable  property.  Finally,  we  pointed  out  how  the  formalization  could  be  used  in  a 
planning  system. 

While  this  paper  is  a  step  towards  showing  that  the  syntactic  approach  can  be  extended 
to  an  adequate  formalization  of  the  interaction  of  knowledge  and  action,  there  is  still  much 
work  to  be  done  in  constructing  a  practical  planner  for  a  multiagent  environment  that 
uses  this  formalism.  Two  areas  in  particular  are  critical.  First,  a  suitable  system  for 
doing  automatic  deduction  in  the  framework  has  to  be  worked  out.  Although  we  have 
advocated  semantic  attachment  as  a  means  of  simplifying  proofs,  we  have  not  yet  explored 
the  problem  of  controlling  a  deduction  mechanism  that  uses  this  technique.  The  second 
area  also  involves  control  issues:  how  can  a  planner  be  designed  to  search  the  space  of 
multiagent  possible  plans  efficiently?  One  of  the  ideas  suggested  by  this  paper  is  to  derive 
lemmas  of  the  form  of  LT3  that  show  the  effect  of  actions  on  an  agent’s  beliefs.  With 
such  lemmas,  a  planning  system  would  have  already  compiled  the  necessary  results  for 
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constructing  new  belief  states  from  previous  ones. 
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Appendix  A:  The  Wise  Man  Puzzle 


This  is  a  solution  to  a  simple  version  of  the  wise  man  puzzle,  for  whose  statement  we 
quote  from  [McC80j:  A  king  wishing  to  know  which  of  his  three  wise  men  is  the  wisest, 
paints  white  dots  on  each  of  their  foreheads,  tells  them  that  at  least  one  spot  is  white,  and 
asks  each  to  determine  the  color  of  his  spot.  After  a  while  the  smartest  announces  that 
his  spot  is  white  reasoning  as  follows:  “ Suppose  my  spot  were  black.  The  second  wisest  of 
us  would  then  see  a  black  and  a  white  and  would  reason  that  if  his  spot  were  black,  the 
dumbest  would  see  two  black  spots  and  would  conclude  that  his  spot  is  white  on  the  basis 
of  the  king’s  assurance.  He  would  have  announced  it  by  now,  so  my  spot  must  be  white.” 

We  will  simplify  this  puzzle  by  having  the  king  ask  each  wise  man  in  turn  what  color 
his  spot  is,  starting  with  the  dumbest.  The  first  two  say  “no,”  and  the  last  says  that  his 
spot  is  white.  Note  that  in  this  formalization  we  will  not  prove  that  the  first  two  wise  men 
don't  know  the  color  of  their  spots,  but  will  take  it  as  given;  the  deduction  of  such  forms 
of  non-knowledge  is  a  hard  problem  that  is  not  addressed  here. 

In  formalizing  the  puzzle,  we  will  take  the  three  wise  men  to  be  Ao,  At,  and  A%,  in 
order  of  increasing  stupidity.  We  will  reason  about  the  puzzle  from  Ao’s  point  of  view,  and 
show  that  Ao  knows  that  his  spot  is  white  after  hearing  the  replies  of  the  other  two.  We 
will  not  be  concerned  with  the  axiomatization  of  the  speech  act  performed  by  the  agents; 
it  will  be  assumed  that  Ao’s  model  of  the  world  changes  appropriately  to  reflect  this  new 
information. 

There  are  three  situations  in  the  puzzle:  the  initial  situation  So,  the  situation  Sj  just 
after  A<i  speaks,  and  the  situation  S 2  just  after  At  speaks.  The  frame  axioms  for  these 
situations  are  simply  that  every  agent  knows  what  he  knew  in  the  previous  situation;  these 
frame  axioms  are  common  knowledge. 

We  will  identify  Ao  with  the  ML,  so  the  goal  is  to  show: 


H(S2,W{A*)) 


M 


in  the  ML.  W(a)  is  the  predicate  whose  meaning  is  “a’s  spot  is  white.”  The  initial  conditions 
of  the  problem  are: 

(1)  W{Al)AW(A2) 

(2)  CFACT{  rW(A0)VW(A,)VW(A2)'1 ) 

(3)  C FACT(  rK(A2,  S0,  ^A*)"1  )VAT(A2,  S9,  "~VK(Ao)n  V ) 

(4)  CFACT(rK(A2,S0,  r^(A1j1)VAT(A2,5o>  r~W(Ai  j1  p ) 

(5)  ^(Aj.So,  ^(AoDv^Aj.So,  ^(AoD 

(6)  CFACT(  r~K(A2,  S0,  rW(A2p  p ) 

(7)  CFACT(  r~AT(Ai ,  Sx ,  rW(At  j1  )n ) 

Line  1  says  that  Ao  observes  white  spots  on  A\  and  A2;  line  2  asserts  that  it  is  common 
knowledge  that  at  least  one  spot  is  white.  The  next  two  lines  state  that  it  is  common 
knowledge  that  A2  can  observe  whether  the  other  two  agent’s  spots  are  white  or  not.  Line 
5  says  that  A\  knows  the  color  of  Ao’s  spot.  And  the  last  two  lines  express  the  effect  of  the 
first  two  agent’s  answers  to  the  king  on  everyone’s  knowledge.  This  axiomatization  will  be 
sufficient  to  prove  that  Ao  knows  his  spot  is  white  in  S2. 

The  first  step  in  the  proof  is  to  show  that  Aj  knows,  in  situation  S\,  that  either  his 
own  or  Ao’s  spot  is  white;  this  by  reasoning  about  A2’s  answer  to  the  king.  We  will  attach 
to  Ai’s  theory  in  situation  Sx  (that  is,  fA«(Ai,5i)),  and  do  our  reasoning  there: 


Ai ’s  Theory  in  Situation  Si 


(8)  ~K(A2,S0,rW(A2T)  6,semai 

(9)  Ar(A2,5o,rW(Ao)VWr(A1)VVT(A2r)  3,semai 

(10)  Ar(A2,50,r(~^(Ao)A~VK(A,))D^(A2r)  9 

(11)  K(A2,S0,r~W(Ao)A~W(Ai)')DK(A2,So,rW(A2)])  10,  MP 

(12)  ~~/C(A2,$0,  r~W(Ao)A~M^(Aip)  8,11  coi 


6, semantic  attachment 


3, semantic  attachment 


8,11  contrapositive 
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Id  these  lines,  we  have  used  the  fact  that  everyone  knows  that  everyone  knows  common 
knowledge  assertions.  At  line  12,  Ai  realizes  that  A2  doesn’t  know  that  both  Ao  and  A\ 
lack  white  dots;  if  he  did,  he  would  have  announced  the  fact. 


Now  Ai  uses  the  common  knowledge  that  A2  can  observe  the  color  of  Ao’s  and  Ai’s 
dots  to  reason  that  one  of  the  latter  has  a  white  dot: 


Ai’s  Theory  in  Situation  S\ 


(13) 

(14) 

(15) 

(16) 

(17) 

(18) 

(19) 

(20) 

(21) 

(22) 

(23) 

(24) 


K(A2,  So,  r~W(Ao)n  )AK(A2,  S0,  ^W^)1 ) 

[13]:  assumption 

K(A2,  So,  r~W(Ao)A~W(Ai)n) 

[13]:13  ,PR 

~K(A2,  So,  r~W(A0f1)  V  ~K(A2,S0,  r~W(A,)n) 

13;  12,14  contradiction 

~tf(A2,S„,  r~W(Ao)n) 

[l6]:assumption 

K(A2,  So,  rW(AoV )VK(A2, So,  r~W(Ao? ) 

3, semantic  attachment 

K(A2,S0,rW(AoT) 

[16]:16,17 

~K(A2,Sq,  r-W(A,D 

[l9]:assumption 

K(A2,S0,  rW(A, p )\/K(A2, S0,  r~W(A,D 

4 .semantic  attachment 

AT(A2,5o,rW(Air) 

[19]:19,20 

K(A2,  So,  rW(Ao)n  )VK(A2,  So,  rW{Ax  )n ) 

15,16,18,19,21 

//(5o,rW(Ao)VW(A,r) 

22,  B2 

W(Ao)VW(Ai) 

23, frame  axioms  ,f?l 

We  first  show  here  that  A2  doesn’t  know  Ao’s  spot  is  black,  or  he  doesn’t  know  that  At’s 
spot  is  black  (line  15).  Assertions  that  follow  from  assumptions  are  indicated  by  a  square 
bracketing  of  the  assumption  line  number  in  their  justification.  Next  we  do  an  analysis 
by  cases  of  line  15;  in  either  case,  line  22  holds:  A2  either  knows  Ao’s  spot  is  white,  or  he 
knows  Ai’s  spot  is  white.  From  this  Ai  concludes  that  either  he  or  Ao  has  a  white  spot 
(line  24).  Note  that  the  frame  axioms  were  needed  to  show  that  the  W  predicate  doesn’t 
change  from  situation  So  to  situation  Si. 


At  this  point  we  are  through  analyzing  Aj's  theory  of  situation  Si,  and  go  back  to  the 
ML  to  reason  about  situation  S2.  By  line  5,  Ai  knows  the  color  of  Ao’s  dot,  so  we  assume 


that  he  knows  it  is  black: 


At  the  Metalevel 

(25)  AT(A,,So,r~VK(Aor) 

(26)  K(A\,S\,  r~fV’(A0j1) 

(27)  fC(A„S,,r~W(Ao)DW(Al)1) 

(28)  K(Ai,Si,  rW(Aif) 

(29)  ~K'(Ai,Si,  rW(Ai)n) 

(30)  ~fr(A,,S0,r~WrMof') 

(31)  K{AlfSo,rW(Ao?) 

(32)  H(S0,  ^(Ao)"1) 


[25]:assumption 
[25]  :25, frame  axioms 

24,  frame  axioms 
[25]:26, 27,  MP 

7,  common  knowledge 

25,  28, 29, contradiction 
5,30 

31, £2 


Under  the  assumption  that  Ai  knows  Ao's  spot  is  black,  we  derive  the  contradiction  of 
lines  28  and  29.  Therefore,  by  line  5,  it  must  be  the  case  that  A\  knows  Ao’s  spot  to  be 
white.  This  is  the  conclusion  of  line  32;  since  this  is  one  of  Ao’s  beliefs,  we  are  done. 


4.  Planning  Natural-Language  Utterances 


This  section  was  written  by  Douglas  Appelt.  This  research  was  supported  in  part  by  the 
Office  of  Naval  Research  under  contract  N 0014-80-  C-0S96  and  in  part  by  the  National  Science 
Foundation  under  grant  MCS-8115105. 

4.1.  Introduction 

This  paper  describes  recent  research  on  a  natural-language  generation  system  that 
is  based  on  planning.  The  view  of  language  production  adopted  here  is  similar  to  that  of 
Allen  [AI180],  Cohen  and  Perrault  [Coh79],  namely  that  speakers  produce  utterances  with  the 
intention  of  satisfying  particular  goals,  and  that  a  hearer's  understanding  of  an  utterance 
depends  on  how  he  interprets  the  utterance  as  fitting  in  with  what  he  believes  to  be  the  speaker’s 
plan. 

A  system  named  KAMP  (for  Knowledge  And  Modalities  Planner)  has  been  developed 
that  plans  natural-language  utterances,  starting  with  a  high-level  description  of  the  speaker’s 
goals.  The  system  can  be  viewed  as  an  extension  of  speech-act  planning  research  by  Cohen 
[Coh79],  but  while  Cohen  stopped  at  producing  abstract  descriptions  of  speech-acts,  KAMP 
extends  the  planning  down  to  the  level  of  the  production  of  English  sentences,  integrating  both 
physical  and  linguistic  actions  that  satisfy  discourse,  knowledge-state  and  referring  goals  into 
utterances  that  achieve  multiple  goals  simultaneously. 

This  research  has  addressed  the  following  three  major  problems: 

*  Developing  a  domain-independent  multiple-agent  planning  system 
called  KAMP. 

*  Extending  the  possible-worlds  semantics  representation  of  proposi¬ 
tional  attitudes  developed  by  Moore  [Moo80j  to  handle  mutual  knowledge 
and  wanting. 


Satisfying  Multiple  Goals  with  a  Request 


Capturing  linguistic  knowledge  in  the  axioms,  critics,  and  procedures 
used  by  KAMP  to  facilitate  the  planning  of  lingui$tic  actions. 

4.2.  Why  Plan  Utterances? 

Figure  1  illustrates  a  typical  situation  arising  when  two  people  cooperate  on  a  common 
task  in  which  a  speaker  plans  an  utterance  that  has  multiple  effects  on  the  intended  hearer. 
The  speaker  points  to  one  of  the  tools  on  the  table  and  says  “Use  the  wheelpuller  to  remove 
the  flywheel.”  The  hearer,  who  is  observing  the  speaker  while  he  makes  the  request,  and 
knows  that  the  speaker  is  pointing  to  the  particular  tool,  thinks  to  himself,  uAh,  so  that’s  a 
wheelpuller.  I  was  wondering  how  I  was  going  to  get  the  flywheel  off.” 


The  speaker’s  utterance  in  figure  1  is  syntactically  very  simple,  but  a  surprising  amount 
of  sophisticated  reasoning  is  required  for  a  speaker  to  produce  such  an  utterance  and  know 
that  it  will  have  the  intended  effect  on  the  hearer.  Most  obviously,  the  speaker  wants  to  make 
a  request  of  the  hearer  to  do  something.  However,  before  he  can  make  the  request,  he  has  to 
determine  whether  the  hearer  has  enough  knowledge  to  carry  it  out.  If  not,  then  the  speaker 
has  to  know  that  the  hearer  can  form  a  plan  for  acquiring  the  knowledge,  or  he  must  furnish 
the  knowledge  himself.  In  this  example,  the  speaker  informs  the  hearer  that  he  should  use  the 
wheelpuller  as  part  of  the  same  utterance  that  he  uses  to  request  that  he  perform  a  removing 
action. 

The  speaker  uses  the  noun  phrase  “the  wheelpuller”  to  refer  to  a  particular  tool.  In 
figure  1,  it  is  evident  from  the  hearer’s  reaction  that  he  didn’t  know  what  a  wheelpuller  is,  and 
the  speaker  knew  that,  because  he  performed  a  pointing  action  to  make  his  intention  to  refer 
clear.  Although  the  speaker  knew  that  the  hearer  did  not  know  what  a  wheelpuller  was,  he 
knew  that  the  hearer  would  know  after  understanding  the  sentence.  The  utterance  of  figure  1 
also  serves  to  inform  the  hearer  that  the  object  the  speaker  is  pointing  to  is  a  wheelpuller.  In 
order  for  the  speaker  to  make  that  inference,  he  had  to  know  that  the  hearer  would  know  that 
he  didn’t  intend  the  object  he  was  pointing  at  to  be  the  referent  of  “the  flywheel.”  He  knows 
that  because  he  knows  the  hearer  knows  that  the  flywheel  is  not  a  tool,  and  therefore  cannot 
fill  the  instrument  role  of  “remove.”  Under  different  circumstances,  the  speaker  could  point  to 
the  flywheel,  utter  the  identical  sentence  as  in  figure  1,  and  reason  that  the  utterance  would 
inform  the  hearer  that  the  object  he  was  pointing  to  was  the  flywheel. 

4.3.  The  KAMP  Language  Planning  System 

It  is  clear  from  the  above  example  that  a  model  of  language  production  that  is  simply 
a  transducer  from  a  logical  form  to  a  surface  utterance  is  not  sufficient  to  account  for  the  way 
that  people  use  utterances  to  satisfy  multiple  goals  —  an  ability  that  requires  the  speaker  and 
hearer  to  make  inferences  about  each  other's  plans.  Furthermore,  utterances  do  more  than  alter 


The  Organization  of  a  Language  Planning  System 


the  participants’  knowledge  and  wants.  They  influence  the  participants’  emotional  attitudes, 
and  affect  the  state  of  the  ongoing  discourse.  Utterances  can  be  planned  with  intentions  of 
acheiving  goals  along  these  dimensions  as  well.  Therefore,  instead  of  a  simple  transducer  from 
logical  form,  KAMP  is  organized  like  the  planner  in  figure  2. 

The  overall  organization  of  KAMP  as  a  hierarchical  planner  similar  in  overall  organiza¬ 
tion  to  Sacerdoti’s  NOAH  [Sac77]  was  discussed  in  detail  in  Appelt  [App80]  [App82|.  KAMP  has 
two  descriptions  of  actions  at  each  level  in  the  action  hierarchy:  A  full  axiomitization  in  terms 
of  possible  worlds,  and  a  shorter,  more  intuitive  description  called  an  action  tummary.  Kamp 
uses  the  action  summaries  as  a  heuristic  device  to  propose  plans  that  it  then  verifies  using  the 


Figure  3 

A  Hierarchy  of  Actions  Related  to  Language 


possible  worlds  axiomatization.  The  heuristic  plan  generation  process  is  implemented  by  the 
NOAH-like  hierarchical  planner,  while  the  verification  process  is  implemented  by  a  first-order- 
logic  theorem-prover. 

Figure  3  illustrates  the  hierarchy  of  actions  that  is  used  by  KAMP  to  plan  linguistic  ac¬ 
tions.  The  central  problem  of  building  a  language-planning  system  around  KAMP  is  formulating 
the  correct  axioms,  and  incorporating  the  correct  action  summaries  and  critic  procedures  into 
KAMP  that  describe  the  actions  of  the  hierarchy  in  Figure  3. 
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4.4.  Axiomatizing  Knowledge  about  Intensional  Concepts 


Axiomatizing  the  actions  of  Figure  3  requires  the  ability  to  specify  how  performing 
actions  affects  the  knowledge,  mutual  knowledge  and  wants  of  agents.  Moore’s  possible-worlds- 
semantics  approach  [M008O]  solves  this  problem  with  respect  to  knowledge  and  its  relation  to 
action.  It  is  impossible  to  describe  Moore’s  approach  in  detail  here,  but  the  central  idea  is 
to  axiomatize  the  possible-worlds  semantics  of  a  modal  object  language  in  a  first-order  meta¬ 
language.  Thus,  the  semantics  of  a  statement  like  Know(A,  P)  is  represented  as  UP  is  true  in 
every  possible  world  consistent  with  what  A  knows.” 

It  is  necessary  to  reason  about  mutual  knowledge  (i.e.,  knowledge  that  A  knows  that 
B  knows  that  A  knows  ...  ad  infinitum)  to  plan  referring  expressions  (see  Clark  &MarshaIl, 
[Cla81]).  KAMP  reasons  about  the  mutual  knowledge  shared  by  two  agents  by  reasoning  about 
what  is  true  in  the  union  of  the  sets  of  possible  worlds  consistent  with  two  agents’  knowledge. 
An  “agent”  called  the  kernel  of  A  and  B  is  defined,  for  whom  the  worlds  consistent  with 
his  knowledge  are  precisely  that  union.  This  is  a  generalization  of  the  “any  fool”  approach 
advtvsted  by  McCarthy  and  Sato  [McC78]. 

Wanting  is  represented  in  KAMP  by  a  relation  between  an  agent  and  a  set  of  possible 
worlds  called  a  situation.  The  situation  is  a  set  of  possible  alternatives  to  the  current  world 
which  an  agent  is  said  to  want.  The  situation  that  an  agent  wants  can  be  characterized  by 
different  propositions  according  to  what  he  knows.  An  agent  is  said  to  want  P  if  there  is  some 
situation  he  wants  such  that  P  is  true  in  every  possible  world  that  is  a  member  of  the  situation, 
with  the  terms  of  P  evaluated  with  respect  to  the  agent’s  knowledge.  This  representation  allows 
one  to  make  a  connection  between  knowledge  and  wanting,  which  while  ignoring  many  of  the 
subtle  problems  associated  with  wanting  and  intention,  is  adequate  for  solving  many  planning 
problems  that  arise  in  the  task-oriented  domains  under  considertation.  It  allows  one  to  reason, 
for  example,  that  if  John  wants  to  meet  the  president  of  the  United  States,  and  if  John  knows 
that  the  president  of  the  United  States  is  Ronald  Reagan,  then  John  wants  to  meet  Ronald 
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4.5.  Axiomatizing  Linguistic  Actions 


As  illustrated  in  figure  3,  the  most  abstract  linguistic  actions  in  KAMP’s  hierarchy 
are  illocutionary  acts.  These  are  actions  such  as  informing,  requesting,  promising,  thanking, 
etc.,  that  can  be  characterized  as  communicative  acts  independent  of  any  particular  linguistic 
realization. 

Speakers  do  not  perform  illocutionary  acts  directly,  but  rather  perform  them  by  means 
of  the  performance  of  surface  speech  acts.  When  a  speaker  plans  a  surface  speech-act,  he  selects 
the  propositional  content  of  the  sentence  he  is  going  to  utter  (which  may  be  different  from  the 
propositional  content  of  the  illocutionary  act  in  the  case  of  indirect  speech  acts),  and  selects 
a  particular  syntactic  structure  that  is  to  be  used  for  the  realization  of  the  illocutionary  act. 
A  fundamental  choice  made  at  this  level  is  whether  to  use  an  imperative,  interrogative  or 
delcarative  sentence.  Each  surface  speech-act  has  a  syntactic  structure  tree  associated  with  it 
that  evolves  as  the  plan  is  expanded  to  include  more  constituents  expanded  to  progressively 
lower  levels. 

The  relationship  between  illocutionary  acts  and  surface  speech-acts  is  similar  to  the 
relationship  between  walking  across  the  room  and  a  sequence  of  muscle  movements.  One  action 
is  performed  by  means  of  performing  the  others.  What  distinguishes  this  relationship  from 
walking  and  muscle  movements  is  that  the  particular  illocutionary  act  that  is  being  performed 
depends  on  the  hearer’s  recognition  of  what  the  speaker  is  trying  to  do.  A  particular  surface 
speech-act,  for  example,  “Can  you  reach  the  tool  on  the  top  shelf?”  can  in  one  case  be  a  request 
to  retrieve  a  tool,  and  in  another  case  a  request  to  inform  the  speaker  of  the  ability  to  retrieve 
a  tool. 

As  the  example  illustrates,  it  is  not  even  the  case  that  there  is  a  one-to-one  correspon¬ 
dence  between  illocutionary  acts  and  surface  speech  acts.  In  this  case  several  informing  and 
requesting  actions  are  being  performed  as  part  of  a  single  surface  speech-act. 


Kamp  has  an  axiomatization  of  each  illocuationary  act  and  surface  speech-act  it  knows 
about  in  terms  of  the  possible  worlds  approach  outlined  above.  This  paper  will  not  describe 
the  axioms  in  detail,  so  the  interested  reader  is  refered  to  [App80]  for  more  information.  The 
general  approach  to  axiomatizing  illocutionary  acts  is  to  describe  only  what  Austin  [ref]  refers  to 
as  illocutionary  effects,  and  not  perlocutionary  effects.  In  other  words,  the  effect  of  informing  a 
hearer  that  P  is  not  that  the  hearer  then  believes  P,  but  that  the  hearer  knows  that  the  speaker 
wants  him  to  know  that  the  speaker  believes  P.  However,  the  speaker  wanting  the  hearer  to 
know  that  the  speaker  believes  P  is  a  reasonable  precondition  of  the  sincere  performance  of 
an  informing  action.  Therefore,  the  effects  of  an  illocutionary  act  can  be  stated  as  producing 
the  mutual  knowledge  between  the  speaker  and  the  hearer  that  the  act  has  been  performed. 
All  deductions  about  the  change  in  the  knowledge  of  the  participants  follow  from  knowing 
the  action  has  been  performed,  and  their  mutual  knowledge  of  the  conditions  on  the  action’s 
performance. 

Surface  speech-acts  include  concept  activation  actions  as  part  of  their  realization  on 
the  next  lower  level  of  abstraction.  Concept  activation  actions  describe  referring  at  a  high 
enough  level  of  abstraction  so  that  they  are  not  constrained  to  be  purely  linguistic  actions. 
When  a  concept  activation  action  is  expanded  to  a  lower  level  of  abstraction,  it  can  result  in 
the  planning  of  a  noun  phrase  within  the  surface  speech  act  of  which  the  concept  activation  is 
a  part,  and  physical  actions  such  as  pointing  that  also  communicate  the  speaker’s  intention  to 
refer,  and  may  be  realized  by  a  plan  that  includes  either  physical  or  linguistic  actions. 

Although  concept  activation  actions  can  be  realized  through  physical  actions,  the  plan¬ 
ner  must  reason  about  their  interaction  with  the  linguistic  actions  being  planned.  Therefore, 
concept  activation  actions  are  expanded  with  an  intention-communication  component  that 
communicates  the  speaker’s  intention  to  refer,  and  a  linguisic-realization  component  that  is 
part  of  the  surface  speech-act  and  takes  into  account  the  grammatical  rules. 

The  lowest  level  actions  of  Figure  3  are  the  utterance  acts.  Utterance  acts  consist  of  the 
utterance  of  particular  sequences  of  words.  The  component  of  KAMP  that  produces  utterance 


acts  from  a  plan  of  hierarchical  linguistic  actions  and  the  constituent-structure  trees  associated 
with  surface  speech  acts  is  quite  simple  because  at  this  point,  no  modifications  to  the  plan 
are  made  because  decisions  are  completely  determined  by  grammatical  rules,  and  there  is  no 
room  for  the  speaker's  intentions  to  influence  the  process.  The  final  stage  of  planning  consists 
primarily  of  making  obligatory  modifications  required  by  the  grammar,  such  as  subject-verb 
agreement,  proper  auxilliary  affixes,  insertion  of  reflexive  pronouns,  etc. 

4.6.  Conclusion 

The  development  of  KAMP  has  been  the  first  step  toward  a  theory  of  planning  natural 
language  utterances  that  allows  the  satisfaction  of  multiple  goals  in  a  single  surface  utterance, 
that  plans  utterances  tailored  to  the  specific  knowledge  of  an  intended  hearer  as  well  as  the 
context  of  the  discourse,  and  that  provides  for  the  integration  of  physical  and  linguistic  actions. 

There  are  a  number  of  areas  in  which  the  concepts  developed  in  KAMP  can  be  profitably 
applied  and  extended.  One  major  area  is  the  planning  of  extended  discourse.  Currently, 
KAMP  plans  only  very  simple  dialogues.  It  may  plan  more  than  one  utterance  if  it  wants  to 
perform  several  illocutionary  acts,  and  it  cannot  figure  out  a  way  in  which  one  can  subsume  the 
others.  The  resulting  dialogues  will  be  coherent  because  the  illocutionary  acts  are  naturally  tied 
together  by  being  part  of  the  same  plan.  However,  to  move  beyond  simple  dialogues  consisting 
of  alternating  one  or  two  sentence  turns,  more  complex,  abstract  discourse-level  actions  must 
be  defined.  McKeown  [McK80j  incorporates  such  strategies  in  a  language  generation  system, 
and  such  actions  need  to  be  formalized  in  a  planning  framework  to  be  used  by  a  system  like 
KAMP. 

KAMP  currently  keeps  track  of  discourse  focus  primarily  so  it  can  generate  appropriate 
referring  expressions.  When  planning  an  extended  discourse,  the  planner  would  also  be  con¬ 
cerned  about  the  speeaker's  need  to  inform  the  hearer  of  topic  shifts.  Topic  shifting  actions, 
similar  to  those  described  by  Reichman  [Rei78],  must  be  formalized  and  planned  when  ap¬ 
propriate. 


The  primary  focus  of  research  on  KAMP  has  been  on  planning  natural-language  ut¬ 
terances.  However,  KAMP  is  a  general  tool  that  can  serve  as  the  basis  of  multiple-agent  plan¬ 
ning  systems  in  a  variety  of  domains.  There  are  many  problems  concerning  planning  to  acquire 
knowledge,  cooperation  among  several  agents  with  limited  resources,  etc.,  for  which  KAMP 


seems  useful. 
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